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Abstract 


The  effect  of  indirect  vision  systems  on  target  detection  and 
recognition  is  of  interest  to  designers  of  future  combat  vehicles.  In  a 
field  study,  eight  participants  detected  and  identified  "pop-up" 
targets  on  an  outdoor  experimental  facility  from  a  stationary  and  a 
moving  vehicle  while  using  a  head-mounted  display  (HMD)  and 
direct  viewing  as  a  control,  with  and  without  sound  localization.  A 
head-slaved  camera  mounted  on  top  of  the  vehicle  provided  the 
image  to  the  HMD.  With  sound  localization  provided  by  localized 
auditory  cueing,  the  computer-controlled  audio  tones  (intermittent  at 
a  0.5-second  interval)  appeared  to  originate  from  the  location  of  the 
target. 

The  results  are  that  more  targets  were  detected  with  direct  viewing 
than  with  the  HMD  and  from  the  stationary  position  than  the  moving 
vehicle.  Although  more  targets  were  detected  with  direct  viewing  from 
the  stationary  vehicle  without  cueing,  sound  localization  improved 
target  detection  in  all  other  treatments.  Similar  comments  apply  to 
identification  from  the  stationary  position;  however,  fewer  targets 
were  identified  from  the  moving  vehicle  with  sound  localization  than 
without.  The  advantages  of  auditory  cueing  for  target  detection  may 
have  been  limited  by  the  choice  of  an  intermittent  tone  for  sound 
localization  and  the  restricted  search  sector  used  in  the  experiment. 

Ratings  of  task  attention  loading  show  that  increased  attention  was 
needed  for  the  detection  task  with  sound  localization,  especially  with 
the  HMD.  This  is  also  true  for  the  identification  task.  Workload  test 
battery  ratings  show  a  significant  increase  in  perceived  workload  with 
the  HMD  as  compared  to  the  direct  vision.  A  test  of  situational 
awareness  shows  significant  effects  with  decreasing  trends  in 
perceived  stability  and  familiarity  and  an  increased  need  to 
concentrate  when  one  is  using  the  HMD  in  the  moving  vehicle. 
Furthermore,  most  of  the  participants  reported  a  general  discomfort 
associated  with  motion  sickness  while  in  the  moving  vehicle  with  the 
HMD.  There  was  no  significant  change  in  heart  rate  with  test  mode, 
suggeshng  that  there  was  no  impact  of  metabolic  work  on  their 
performance.  As  a  result  of  the  study,  the  participants  rated  direct 
viewing  as  more  useful  than  the  HMD  and  the  sound  localization  as 
being  of  more  use  with  the  HMD  than  with  direct  viewing. 
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DETECTING  TARGETS  FROM  A  MOVING  VEHICLE  WITH  A  HEAD- 
MOUNTED  DISPLAY  AND  SOUND  LOCALIZATION 


1.  Introduction 


In  this  section  of  the  report,  we  describe  the  background  rationale  for  the 
experiment,  and  we  comment  on  the  effects  of  camera  and  display  selection  on 
performance  as  well  as  the  benefits  of  sound  localization  for  target  detection.  We 
suggest  criteria  for  optimal  crew  performance,  including  workload  and 
situational  awareness  (SA),  and  we  discuss  the  potential  effects  of  motion 
sickness. 

1.1  Background 

The  Army  needs  combat  vehicles  that  are  smaller,  lighter,  more  lethal,  survivable, 
and  more  mobile  to  support  a  rapidly  deployable  force.  Combined  with  the  need 
to  assimilate  and  distribute  more  information  to,  from,  and  within  the  vehicle  as 
the  Army  moves  toward  a  digital  battlefield,  there  is  the  need  for  an  increase  in 
vehicle  and  command,  control,  communications,  computers,  and  intelligence 
systems  integration  and  performance.  Consequently,  the  Army  will  need 
sophisticated,  highly  integrated  crew  stations  for  these  future  combat  vehicles.  In 
support  of  this  effort,  the  U.S.  Army  Tank  Automotive  Development  and 
Engineering  Center  (TARDEC)  is  developing  the  TARDEC  Crew  Integration  and 
Automation  Test  Bed  (CAT)  Advanced  Technology  Demonstrator  (ATD).  The 
purpose  of  the  CAT  ATD  is  to  demonstrate  crew  interfaces,  automation,  and 
integration  technologies  required  to  operate  and  support  future  combat  vehicles. 
The  U.S.  Army  Research  Laboratory  (ARL)  is  providing  human  factors  expertise 
in  determining  the  effect  of  these  new  crew  station  technologies  with  a  continuing 
series  of  studies  and  investigations.  The  results  can  dramatically  increase  the 
operational  effectiveness  and  capabilities  with  fewer  crew  members,  thereby 
contributing  to  smaller  and  lighter  weapons  systems.  In  this  study,  ARL 
conducted  an  experiment  to  determine  the  target  detection  and  identification 
performance  of  vehicle  crews  who  are  using  computerized  indirect  vision  with 
head-mounted  displays  (HMDs)  and  optical  systems. 

1.2  Indirect  Vision  System 

To  satisfy  the  Army  requirements  for  reduced  gross  weight  and  lower  silhouette, 
as  well  as  the  need  for  increased  crew  protection  against  ballistic  and  directed 
energy  threats,  designers  of  future  armored  combat  vehicles  will  place  the  crew 
stations  deep  within  the  hull  of  the  vehicle.  For  protection  against  direct  and 
indirect  fire  as  well  as  chemical  and  biological  agents,  the  crews  will  operate  with 
their  hatches  closed  and  sealed.  High  intensity  combat  lasers  that  can  penetrate 
direct  vision  blocks  may  force  the  crew  to  operate  on  the  battlefield  in  a 
"buttoned  up"  mode,  with  indirect  vision  systems  for  driving  and  target  search 
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and  engagement.  The  conventional  optics  consisting  of  periscopic  vision  blocks 
and  optical  sights  will  be  replaced  by  electronic  displays  at  each  crew  station 
and  by  externally  mounted  camera  arrays  on  the  vehicle.  These  vision  systems 
will  show  computerized  digital  images  that  are  electronically  acquired  by  the 
camera  arrays.  The  crew  member  will  see  a  selected  portion  of  the  computerized 
display  buffer  that  depends  on  his  or  her  role  and  viewing  direction.  No  doubt, 
by  the  incorporation  of  virtual  reality  display  and  camera  components,  future 
vision  systems  will  appear  to  have  a  "see-through  armor"  capability  with  a 
seemingly  direct  view  of  the  external  scene  through  the  armored  hull  of  the 
vehicle. 

1.3  Camera  Selection 

Before  indirect  vision  systems  can  be  considered  for  future  vehicle  designs, 
combat  and  materiel  developers  will  need  to  know  the  potential  impact  on  the 
crew's  combat  performance.  During  night  operations,  replacing  the  vision  blocks 
with  infrared  thermal  viewers  improves  the  crew  performance  by  enhancing 
visibility  at  low  light  levels.  In  daylight  conditions,  however,  the  use  of  indirect 
vision  may  cause  a  reduction  in  visual  performance  and  SA.  This  is  because  of 
the  decrease  in  visual  resolution  and  field  of  view  (FOV)  of  the  current  sensors 
and  displays  as  compared  to  vision  of  the  human  eye  through  vision  blocks.  This 
reduction  in  visual  performance  may  reduce  overall  combat  performance.  Further, 
the  choice  of  camera  configuration  and  placement  on  the  vehicle  can  influence 
performance.  For  example,  a  single  telescopic  camera  used  in  target  search  and 
detection  may  not  have  the  performance  of  a  multiple  array  of  such  cameras 
placed  about  the  vehicle  and  electronically  integrated  to  give  a  more  naturally 
panoramic  view.  In  the  limit,  the  panoramic  view  would  approach  direct  vision 
as  the  desired  case. 

1.4  Display  Selection 

The  display  selection  will  influence  the  vehicle  design.  The  design  may  use  a  set 
of  panel-mounted  displays,  either  cathode  ray  tube  (CRT)  or  flat  panel  liquid 
crystal  displays  (LCDs),  which  are  fixed  in  a  panoramic  arrangement  about  the 
crew  member  s  station.  Another  option  is  the  use  of  a  miniature  HMD  attached 
to  the  crew  member's  helmet.  The  display  scene  of  the  HMD  can  be  "slaved"  to 
head  movements  with  a  head  tracker  and  for  that  reason,  may  appear  more 
natural  but  with  a  limited  FOV.  Compared  to  the  panel  CRT  and  LCDs,  the  use 
of  the  HMD  significantly  reduces  the  size,  weight,  and  power  requirements  for 
the  crew  station.  However,  the  miniature  displays  that  are  currently  available 
cannot  match  the  brightness  and  resolution  of  the  larger  panel  systems  and  may 
result  in  degraded  crew  performance.  The  display  selection  will  also  have  an 
impact  on  the  crew  size  needed  to  operate  the  armored  vehicles  of  the  future.  We 
can  expect  in  the  future  to  have  two-  or  three-person  crews  available.  The  form 
of  computerized  aiding  used  with  the  crew  member's  electronic  associate1  for  the 
armored  crew  station  is  influenced  by  the  display  design.  A  panoramic  design  of 
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panel  displays  for  a  two-person  crew  seated  together  may  facilitate  team 
interaction  and  performance.  In  contrast,  the  use  of  HMDs  may  tend  to  isolate 
the  crew  members  and  may  require  increased  electronic  communication  between 
them. 


1.5  Sound  Localization 

One  problem  of  interest  is  the  effectiveness  of  an  HMD  in  target  detection  and 
identification  with  a  head-controlled  telescopic  sight.  A  problem  for  target 
detection  and  identification  with  indirect  vision  systems  is  the  inherently  low 
visual  acuity  and  limited  FOV  provided  by  the  current  display  technology.  On 
the  modem  digital  battlefield,  the  crew  may  have  advanced  knowledge  of  the 
target's  location.  In  this  case,  auditory  directional  cueing  by  sound  localization 
should  facilitate  visual  target  detection  by  orienting  the  gunner  toward  the  target. 
The  choice  of  auditory  cueing  for  a  visual  task  reduces  cognitive  interference.  This 
is  shown  by  the  literature  about  interference  for  performance  of  concurrent  tasks 
with  directed  attention,  time  sharing,  and  workload  (Wickens,  1992).  Cueing  can 
act  as  a  secondary  task,  either  supporting  or  interfering  with  the  primary  task, 
depending  on  the  display  format  design.  Wickens  (1992)  reports  poor  time 
sharing  for  two  visual  displays  so  spatially  separated  that  both  cannot  be 
accessed  by  foveal  vision  simultaneously.  However,  displays  of  different 
modalities  such  as  visual  and  auditory  can  have  reduced  interference  with  task 
performance. 

1.6  Optimal  Crew  Performance 

Another  factor  in  the  display  design  is  the  need  to  maintain  SA  and  a  mental 
model  of  the  task.  As  noted  by  Endsley  (1993),  SA  is  a  precursor  to  optimal 
performance,  since  a  loss  in  awareness  impacts  decision  making  and  leads  to  a 
risk  of  performance  error.  However,  human  operators  can  perform  well  with  less- 
than-optimal  display  systems  by  increasing  their  efforts  to  meet  the  more 
demanding  workload.  The  resulting  increase  in  flow  of  information  and  tasks 
may  result  in  a  loss  of  SA;  this  is  because  the  ability  of  humans  to  process 
information  is  innately  limited.  For  this  reason,  besides  demonstrating  improved 
performance,  a  further  criterion  is  that  the  display  system  should  not  generate 
excessive  workload  or  decrease  SA. 

1.7  Motion  Sickness 

Another  factor  influencing  crew  performance  is  the  possibility  of  motion  sickness, 
which  can  occur  in  an  enclosed  cab  area  with  spatial  disorientation.  As  noted  by 
Yardley  (1992),  motion  sickness  is  provoked  by  sensory  conflict  between  the 
visual  and  sensorimotor  activities,  which  involve  the  vestibular  system  through 
head  movements.  Associated  are  a  constellation  of  mainly  autonomic  symptoms 
such  as  pallor,  drowsiness,  salivation,  sweating,  nausea,  and  finally,  vomiting  in 
the  more  severe  cases  of  motion  sickness.  Although  some  individuals  may 
eventually  adapt  to  situations  that  initially  provoke  sickness  (Yardley,  1992),  the 
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occurrence  may  still  be  severe  enough  to  arrest  task  performance  until  the 
symptoms  subside. 


2.  Objective 


The  objective  was  to  compare  direct  vision  to  an  HMD,  with  and  without  audio 
localizing  cues,  for  target  detection  and  identification  from  a  stationary  and  a 
moving  vehicle.  The  results  are  compared  to  those  for  direct  vision  as 
representative  of  a  perfect  "see-through  armor"  vision  system. 


3.  Methodology 


The  experimental  apparatus,  facility,  and  targets,  questionnaires,  participants, 
experimental  design,  experimental  procedure,  and  participant’s  scenario  are 
reported  here. 

3.1  Apparatus 

The  experimental  apparatus  is  a  Sony  Electronics  EVI-330  charge-coupled  device 
color  camera  with  video  output  to  a  Virtual  i-O  i_glasses  HMD  with  stereo 
earphones.  The  apparatus  is  located  in  a  high  mobility,  multipurpose,  wheeled 
vehicle  (HMMWV)  with  an  on-board  computer.  The  electronic  camera  is 
attached  to  a  computer-controlled  Directed  Perception,  Inc.,  pan-and-tilt 
mechanism  that  is  mounted  on  the  roof  of  the  vehicle.  Figure  1  shows  a  side  view 
of  the  HMMWV  with  the  electronic  camera;  a  global  positioning  system  antenna 
is  to  the  rear  of  the  camera.  Figure  2  shows  a  close  view  of  the  camera  on  the 
pan-and-tilt  mechanism. 

Figure  3  shows  a  participant  sitting  in  the  front  passenger  seat  and  wearing  a 
safety  helmet  with  the  HMD.  A  magnetic  field  sensor  is  attached  to  the  top  of 
helmet  for  head  tracking.  The  participant  wears  stereo  earphones  under  the 
helmet  for  sound  localization  by  localized  auditory  cueing.  He  wears  a  Computer 
Instruments  Corporation  (CIC)  UNIQ  heart  watch,  which  consists  of  a  chest 
band  and  wrist  exercise  recorder  for  heart  rate  data.  The  earphones  are 
acoustically  driven  by  a  computer-controlled  Air  Force  three-dimensional  (3-D) 
Sound  Localization  System  with  input  from  a  tape  recorder. 

Figure  4  shows  a  view  of  the  process  control  computer  in  the  rear  bay  area  of  the 
vehicle  next  to  the  right  rear  passenger  seat  that  is  occupied  by  the  experimenter 
during  the  experiment.  A  magnetic  field  source  is  mounted  above  and  behind  the 
front  seat.  An  Ascension  Technology  Corporation  "flock  of  birds"  tracking 
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processor  provides  the  position  and  orientation  of  the  field  sensor  to  the 
computer,  as  measured  in  the  magnetic  field  of  the  field  source  that  is  fixed  to 
the  frame  of  the  vehicle. 


Figure  1.  HMMWV  Experimental  Vehicle  With  Roof-Mounted  Camera. 
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Figure  2.  Close  View  of  Camera  With  Pan-and-Tilt  Mechanism. 
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Figure  3.  Participant  With  HMD. 


Figure  4.  Experimenter's  Computer  Control  Station. 
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The  right  front  passenger's  seating  position  was  enclosed  during  the  indirect 
vision  portion  of  the  study  to  keep  sunlight  from  "washing  out"  the  HMD. 
Opaque  coverings  were  put  over  the  right  front  windshield  and  the  side  window, 
and  a  wooden  wall  was  placed  between  the  driver  and  the  passenger's  area.  A 
curtain  was  dropped  behind  the  passenger's  seat  to  complete  the  light  shielding. 
During  the  direct  vision  studies,  the  passenger's  area  was  open.  The  right  side 
windshield  and  the  right  front  door  were  removed  from  the  vehicle  to  provide  the 
participant  with  an  unobstructed  view  to  the  right  front  and  side. 

3.1.1  Equipment  Integration 

Figure  5  is  a  schematic  of  the  hardware  used  in  the  experiment.  See  Table  1  for  a 
complete  list  of  the  equipment  used  in  the  study.  As  is  shown  in  Figure  5,  the 
NTS2-170  video  output  of  the  roof-mounted  electronic  camera  is  input  to  the 
HMD,  and  the  participant  sees  what  the  camera  "sees".  Also  fed  by  the  camera 
output  is  a  video  monitor  for  the  experimenter.  The  camera  rotates  with  the 
participant's  head  through  the  computer-controlled  pan-and-tilt  mechanism  with 
input  from  the  head  tracker.  The  tracking  processor  provides  to  the  computer  the 
position  and  orientation  of  the  field  sensor  attached  to  the  helmet,  as  measured 
by  the  magnetic  field  source  fixed  to  the  frame  of  the  vehicle.  The  computer 
output  commands  the  orientation  and  speed  of  movement  of  the  pan-and-tilt 
mechanism  for  optimum  linkage  of  the  camera  to  the  head  direction.  The 
participant  controls  the  FOV  of  the  camera  with  a  manual  zoom  control  that  he 
or  she  holds  in  his  or  her  lap.  As  well  as  controlling  the  camera,  the  FOV  setting 
is  output  to  the  computer  for  reducing  the  tracking  sensitivity  at  increased  zoom 
and  resetting  it  once  zoom  is  removed.  Finally,  the  computer  provides  the 
direction  for  the  localized  auditory  cueing  to  the  3-D  sound  localization  system. 
The  mono-audio  output  of  the  tape  recorder  is  modified  by  the  3-D  sound 
localization  system  to  provide  audio  signals  shifted  in  phase  and  intensity  to  the 
stereo  earphones  provided  for  the  participant  and  the  experimenter.  The  latter 
uses  his  earphones  to  ensure  proper  functionality  of  the  audio  system  during  the 
sound  cueing  portion  of  the  experiment. 

3.1.2  Indirect  Vision  System 

The  visual  characteristics  of  the  indirect  vision  system  are  listed  in  Table  2,  along 
with  those  of  direct  vision  for  comparison.  The  table  shows  that  the  indirect 
vision  system  is  limited  by  the  resolution  and  FOV  of  the  HMD  used  in  this 
study.  At  30  degrees,  the  horizontal  FOV  of  the  HMD  is  61%  of  the  48.8-degree 
FOV  of  the  camera.  Because  of  the  differences  in  FOV  of  the  camera  and  HMD, 
the  scene  on  the  HMD  appears  expanded  1.63  times  that  of  the  real  world, 
making  the  targets  0.61  times  smaller  in  linear  size.  Furthermore,  the  HMD  with 
180,000  (528  x  340)  rasters  has  68.8%  of  the  angular  resolution  of  the  camera 
768  x  494  rasters.  For  these  reasons,  targets  on  the  HMD  appear  0.42  smaller  in 
linear  size  than  they  would  with  an  HMD  optically  matched  to  the  camera.  The 
electronic  camera  is  auto-focusing  with  automatic  white  balance  and  brightness 
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control  by  auto-iris.  The  zoom  for  magnification  is  manually  controlled  and 
desensitizes  the  head  response  to  allow  directional  control  by  the  participant  a  t 
the  higher  magnification. 


audio 
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player 


pan  &  tilt 
controller 


pan  &  tilt 
mechanism 


electronic 
color  camera 


3D 

sound 
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PCx586 


head 

tracker 
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video 

monitor 
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manual  zoom 
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field 
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experimenter’s 
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I  ~1  stereo 

earphones 


Figure  5.  Schematic  of  Equipment  Layout. 


Table  1.  Experimental  Apparatus 


1 .  High  mobility,  multipurpose,  wheeled  vehicle. 

2.  Sony  Electronics'  EVI-330  color  camera. 

3.  Directed  Perception  s  pan-and-tilt,  computer-controlled,  stepper  motor 
mechanism. 

4.  Ascension's  Flock  of  Birds  (magnetic  field)  head  tracker. 

5 .  Virtual  i-O's  i-glasses,  liquid  crystal,  video  game,  HMDs. 

6.  Safety  helmet  with  field  sensor  attached. 

7 .  Manual  control  mechanism  for  color  camera  telescopic  zoom. 

8 .  Industrialized  personal  computer,  x586. 

9 .  Tape  recorder  with  pre-recorded  electronic  sound  source. 

1 0 .  Air  Force  3-D  Sound  Localization  System. 

11.  Stereophonic  ear-plug  phones. 

12.  Video  monitor. 

13.  CIC  s  UNIQ  heart  watch  (model  8799)  exercise  recorder. 

14.  Light  shielding  for  the  windshield,  side  window,  and  passenger's  area. 

15.  Two-way  hand  radio. 

16.  Hand  paddle  for  signaling. 

17.  Hand-held  binoculars  (7X). 
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Table  2.  Vision  System  Properties 


Property 

Camera 

Indirect 

HMD 

Direct 

Binoculars 

Natural 

rasters  (HxV) 

768x494 

528x340 

infinite 

infinite 

FOV  (degs)@lx 

48.8 

30 

76.3 

170 

Magnification 

12x 

lx 

7x 

lx 

FOV  @  mag 

4.1 

30 

10.9 

170 

3.1.3  Sound  Localization  System 

The  acoustic  properties  of  the  auditory  source  used  in  the  sound  localization 
portion  of  the  experiment  are  listed  in  Table  3.  The  input  sound  from  the  tape 
recorder  to  the  sound  localization  system  is  a  sequence  of  100-millisecond  chirps 
(i.e.,  sweep  sinusoids)  presented  at  0.5-second  time  intervals.  The  localization  is 
synthetically  generated  (Wenzel,  1991)  as  localized  auditory  cues  from  the  Air 
Force  3-D  sound  localization  system  by  convoluting  the  input  sound  with  a 
directional  sensitive  head-related  transfer  function  (HRTF)  measured  for  a 
manikin  standard.  The  system  produces  different  stereophonic  sounds  for  each 
ear  through  the  earphones.  The  sound  cue  appears  as  a  stereophonic  sound 
coming  from  the  target  along  the  bearing  to  the  participant.  He  or  she  hears  a 
chirping  sound  coming  roughly  from  the  direction  of  the  target  and  tends  to  turn 
his  or  her  head  toward  the  sound. 


Table  3.  Sound  Source  Properties 


Property 

Value 

Characteristic 

linear  sweep  sinusoid 

Spectrum  range 

500  to  4500  Hertz 

Duration 

100  milliseconds 

Delivery  rate 

2  Hertz 

It  is  interesting  to  note  that  the  literature  (Begault  &  Wenzel,  1990)  about 
synthetic  sounds  generated  in  earphones  reports  that  humans  have  a  roughly 
20-degree  localization  accuracy  in  both  azimuth  and  elevation.  The  localization  is 
reported  to  be  more  accurate  at  a  30-  to  60-degree  angle  to  the  side  of  the  head 
than  at  other  angles.  The  sound  source  tends  to  be  localized  in  a  bow-tie  shape  as 
the  source  is  moved  in  the  horizontal  plane  about  the  head,  with  the  source 
becoming  internalized  for  source  bearings  within  30  degrees  of  the  lateral  plane 
through  the  cranial  midline,  approached  from  either  the  front  or  back. 
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Furthermore,  the  source  tends  to  appear  elevated  for  sources  to  the  front  and 
depressed  below  the  horizontal  plane  for  sources  to  the  rear.  Experiments  have 
shown  that  the  best  localization  occurs  for  sources  placed  at  60  and  150  degrees 
for  the  right  ear  and  210  and  300  degrees  for  the  left  ear  where  the  bearing  is 
measured  in  the  horizontal  plane  from  the  person's  nose  about  the  head  right  to 
left.  It  is  belie\  ed  that  the  internalization  of  a  sound  source  which  is  presented  to 
the  front  is  the  result  of  visual  dominance;  however,  this  may  be  attributable  to 
earphone  design  (Gierlich,  1992;  Moller,  1992).  Reversals  of  apparent  direction 
between  front  and  rear  are  common  (about  11%)  for  sources  near  the  cranial 
midline  (Wightman  &  Kistler,  1989).  This  may  be  because  frontal  sources  not 
seen  are  interpreted  as  being  to  the  rear. 

3.1.4  Vehicle  Computer 

The  computer  program  controlling  the  equipment  integration  was  executed  by  the 
experimenter  at  the  start  of  each  block  of  experimental  runs.  The  program  first 
reads  data  files  and  then  hibernates.  The  data  files  selected  by  the  experimenter 
for  the  block  specify  the  target  set  and  order  of  presentation,  the  locations  of  the 
targets  on  the  experimental  facility,  the  vehicle  mode  (stationary  versus  moving), 
the  starting  location,  and  if  moving,  the  vehicle  speed  and  direction  of  travel.  The 
experimenter  releases  the  program  from  hibernation  with  a  key  push  from  the 
keyboard  at  the  start  of  an  experimental  rim  and  returns  it  at  the  end  with 
another  key  push.  The  rim  number  and  the  designator  and  location  for  the 
corresponding  target  are  displayed  on  the  computer  monitor  at  the  start  of  the 
run  to  verify  the  functionality. 

During  the  sound  localization  trials,  the  auditory  cueing  from  the  sound 
localization  system  is  turned  on  by  the  computer  during  the  trial  run  and  is 
otherwise  suppressed  in  response  to  the  keyboard  entries.  The  direction  for 
sound  localization  is  computed  from  the  target  location  on  the  experimental 
facility,  the  position  of  the  vehicle,  and  the  head  orientation  of  the  participant 
within  the  vehicle.  The  target  location  is  determined  from  the  trial  run  and  the 
files  on  the  target  presentation  order  and  the  locations  of  the  targets  on  the 
facility.  For  the  moving  studies,  the  computer  estimates  the  position  of  the 
vehicle  from  the  starting  position,  the  time  lapsed  since  starting  the  trial,  and  the 
speed  of  travel  along  the  trial  route.  The  head  orientation  is  read  in  real  time 
from  the  head  tracker. 

3.2  Target  Facility 

The  participants  were  evaluated  on  an  outdoor,  computer-controlled,  small  arms 
firing  facility,  which  is  600  meters  long.  The  facility,  known  as  the  small  arms 
shooting  performance  research  facility  (SASPRF),  is  located  at  Aberdeen  Proving 
Ground  (APG),  Maryland,  and  is  used  for  measuring  the  weapon  performance  of 
soldiers.  The  facility  is  permanent  and  consists  of  160  stationary  "pop-up" 
targets  arranged  in  four  lanes  of  40  targets  each,  spaced  at  distances  from 
10  meters  to  550  meters  from  the  firing  line.  Figure  6  shows  an  aerial  view  of  the 
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firing  facility  with  all  targets  in  a  raised  position.  The  figure  shows  a  control 
center  at  the  firing  line  and  an  access  road  paralleling  the  left  side  of  the  facility. 
The  targets  are  operated  by  a  computer  system  in  the  control  center.  Figure  7 
shows  a  view  of  the  control  center.  The  computer  system  can  present 
programmed  arrays  of  targets  at  any  distance,  time  interval,  and  sequence.  The 
system  records  and  reduces  events,  such  as  targets  presented,  target  time,  target 
hits,  and  times  of  hits. 


Figure  6.  Aerial  View  of  the  Target  Facility. 


Figure  7.  View  of  Target  Facility  Control  Center. 
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3.3  Targets 

The  pop-up  targets  consist  of  an  E-type  silhouette  mounted  on  a  pneumatic 
target  mechanism  that  is  capable  of  raising  and  lowering  the  silhouette  under 
control  of  the  facility's  computer.  Figure  8  shows  a  sketch  with  dimensions  of  a 
target  board.  The  target  is  a  "full  trunk"  (18.5  in.  wide  by  29  in.  high)  human 
silhouette  with  an  8.5-  by  10-inch  head.  The  targets  used  in  this  study  are 
colored  olive  drab.  The  targets  stand  roughly  24  inches  above  the  grass  line  at  the 
shoulder.  All  targets  used  in  this  study  have  a  dull  black  Landolt  C  ring  (8.66-in. 
outer  diameter,  1.625-in.  thickness)  painted  on  the  face  for  identification,  with 
the  1.625-inch  ring  gap  oriented  up,  down,  left,  or  right,  in  accordance  with  a 
random  assignment.  The  rings  were  visible  against  the  targets  with  the  binoculars 
and  telescopic  zooming  but  not  with  direct  vision. 


Figure  8.  Target  Silhouette. 


3.4  Participants 

Eight  Department  of  Defense  civilian  and  military  male  volunteers  who  have 
good  eyesight  (nominal  20/20,  normal  or  corrected  by  glasses  or  contacts)  served 
as  participants  in  this  study.  Three  of  the  subjects  were  enlisted  military 
assigned  to  the  Aberdeen  Test  Center,  APG.  Three  subjects  were  military  and 
two  were  civilian  volunteers  assigned  to  ARL.  The  demographics  of  the 
participants  are  listed  in  Table  4  in  order  of  rank,  along  with  the  visual  acuity 
(far  vision)  and  hearing  acuity  as  determined  from  medical  records  and 
measurements  with  a  Titmus  visual  acuity  meter. 


12 


Table  4.  Demographics  of  Participants 


Far  Vision  Hearing  Experience 


Rank 

Age 

MOS 

Education 

Acuity 

Corrected 

Left 

Right 

Military 

Computers 

SPC 

20 

11M 

GED 

20/18 

none 

low 

low 

Bradley 

PC/video 

driver 

games 

SPC 

28 

11M 

HS 

20/22 

none 

tinnitus 

low 

SGT 

24 

11M20 

HS 

20/20 

contacts 

good 

good 

TOW 

PC /video 

operator 

games 

SFC 

36 

54B4H 

BS 

20/20 

glasses 

good 

good 

artillery 

PC 

MAJ 

36 

ORD 

MS 

20/30 

none 

good 

good 

com 

PC  /video 

games 

MAJ 

40 

51A 

MA 

20/20 

glasses 

good 

good 

mech 

PC/video 

units 

games 

GS14 

61 

HFE 

BS 

20/30 

glasses 

good 

good 

PC 

GM14 

47 

HFE 

MS 

20/20 

none 

good 

good 

PC  /video 

games 


TOW  =  tube-launched,  optically  tracked,  wire-guided  (missile) 


3.5  Experimental  Design 

3.5.1  Design 

The  experimental  design  was  a  fixed  factorial  2x2x2  experiment  with  repeated 
measures  on  the  vehicle  speed,  vision  mode,  and  cueing  factor,  and  the 
participants  as  a  random  factor. 

The  independent  variables  are 

1.  two  levels  of  road  speed  (stationary  versus  10  mph), 

2.  two  levels  of  vision  (direct  versus  HMD),  and 

3.  two  levels  of  cueing  (none  versus  sound  localization). 

The  dependent  variables  are 

1.  the  times  to  complete  detection, 

2.  the  target  detection  events  scored  as  a  success  or  failure, 

3.  the  times  for  identification,  given  detection, 

4.  the  target  identification  scores, 

5.  the  correct  identification  scores, 

6.  the  number  of  false  target  detections, 

7.  the  heart  rate,  and 

8.  the  scores  from  a  battery  of  questionnaires. 

3.5.2  Null  Hypothesis 

There  are  no  differences  in  target  detection  and  identification,  false  alarm  rate, 
heart  rate,  or  questionnaire  scores  among  the  fixed  factorials. 
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3.6  Questionnaires 

A  battery  of  questionnaires  was  used  to  measure  the  subjective  effects  of  the 
treatments  on  different  aspects  of  the  task  workload  and  consists  of  the 
following  items: 

3.6.1  Attention  Allocation  Loading 

A  questionnaire  was  used  for  rating  the  allocation  of  attention  to  the  visual, 
auditory,  cognitive,  and  motor  processing  channels  of  the  human  operator 
according  to  loading  factors  (McCracken  &  Aldrich,  1984).  These  loading  factors 
are  used  in  task  analysis  workload  (TAWL)  simulations  (Allender,  Salvi,  & 
Promisel,  1998).  The  questionnaire  (see  Appendix  A)  consists  of  a  set  of  four  7- 
point,  bipolar  scales  for  rating  the  attention  loading  on  each  channel,  with  verbal 
anchors  for  corresponding  activities  overlaid  on  the  scales. 

3.6.2  NASA-TLX  Workload 

The  National  Aeronautics  and  Space  Administration  (NASA)-Task  Loading 
Index  (TLX)  workload  questionnaire  (Hart  &  Staveland,  1988)  was  used  for 
rating  the  perceived  workload  in  terms  of  task  demand  and  interaction.  The 
NASA-TLX  is  a  multidimensional  rating  procedure  for  the  subjective  assessment 
of  workload.  Workload  has  been  defined  as  a  hypothetical  construct,  which 
represents  the  cost  incurred  by  the  human  operator  to  achieve  a  specific 
performance  level.  The  construct  is  composed  of  behavioral,  performance, 
physiological,  and  subjective  components,  which  result  from  the  interaction 
between  a  specific  individual  and  the  demands  imposed  by  a  particular  task. 
The  questionnaire  (see  Appendix  A)  consists  of  six  scales  that  relate  to  the 
demands  imposed  on  a  subject  and  the  interaction  of  the  subject  with  the  task. 
The  Mental,  Physical,  and  Temporal  scales  measure  the  demands,  while  the 
Effort,  Frustration,  and  Performance  scales  relate  to  the  interaction  with  the  task. 

3.6.3  Motion  Sickness 

A  motion  sickness  questionnaire  used  for  the  subjective  estimation  of  motion 
sickness  (Kennedy,  Lilienthal,  Berbaum,  Baltzley,  &  McCauley,  1989).  The 
questionnaire  (see  Appendix  A)  consists  of  4-point,  bipolar  rating  scales 
consisting  of  verbal  descriptors  for  the  level  of  sickness  of  16  symptoms  such  as 
general  discomfort,  eye  strain,  dizziness,  and  nausea,  among  others.  Based  on 
data  from  a  factor  analysis  of  simulator  sickness  experiences,  a  procedure  has 
been  developed  (Kennedy,  Lane,  Lilienthal,  Berbaum,  &  Hettinger,  1992)  for 
reducing  the  scores  to  subscales  for  the  symptomatic  components  of  oculomotor 
stress  (eye  strain),  nausea,  and  disorientation,  and  a  measure  of  total  severity. 

3.6.4  SART  Situation  Awareness 

The  Situation  Awareness  Rating  Technique  (SART)  questionnaire  used  for  rating 
SA  (Taylor,  1988,  1989;  Taylor  &  Selcon,  1994).  The  SART  questionnaire  (Selcon, 
Taylor  &  Koritsas,  1991)  was  designed  to  measure  subjective  ratings  of  non¬ 
attention  factors  such  as  domain  knowledge  or  schemata  and  experience,  the 
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cognitive  nature  of  the  information  received  while  the  task  is  being  performed, 
and  the  workload  needed  to  process  the  information.  The  questionnaire  (see 
Appendix  A)  uses  ten  independent  seven-point  bipolar  dimensions,  which  are,  in 
turn,  classified  into  three  major  domains  of  situation  demand,  supply,  and 
understanding.  The  ten  dimensions  of  the  questionnaire  are  instability, 
variability,  and  complexity  of  the  situation  for  the  demand  domain;  the  arousal, 
spare  mental  capacity,  concentration,  and  division  of  attention  for  the  supply 
domain;  and  the  quantity,  quality,  and  familiarity  of  the  information  for  the 
understanding  domain.  The  questionnaire  is  the  result  of  a  study  by  Taylor 
(1988, 1989)  involving  subjective  ratings  of  bipolar  awareness  constructs  elicited 
from  air  crews  about  their  experience  and  knowledge,  using  a  repertory  grid 
technique.  Taylor  reportedly  found  that  ten  independent  bipolar  constructs 
emerged  from  the  44  constructs  provided  by  the  air  crew,  as  determined  by 
eliciting  frequency,  principal  component  loading,  and  inter-correlation  clustering. 
These  were  reduced  by  Taylor  to  the  ten  dimensions  of  the  SART  questionnaire. 

3.6.5  Exit  Evaluation 

In  addition,  a  utility  rating  questionnaire  was  applied  at  the  end  of  the 
experiment.  A  sample  questionnaire  is  shown  in  Appendix  A. 

3.7  Control  of  the  Experiment- wise  Type  I  Error 

A  problem  in  analyzing  the  data  from  this  experiment  is  the  control  of  the 
experiment-wise  Type  I  (false  positive)  error.  This  is  because  the  chance  that  a 
measure  will  erroneously  prove  to  be  significant  increases  statistically  with  the 
number  of  analyses  performed.  In  all,  105  measures  were  collected  for  each  trial 
run.  These  include  12  target  detection  and  identification  events,  with  each  event 
consisting  of  a  detection  score,  a  detection  time,  an  identification  score,  an 
identification  time,  and  a  correct  identification  score.  In  addition,  four  false 
alarm  scores,  a  heart  rate,  six  workload  questions,  10  SART  questions,  16  motion 
sickness  questions,  and  two  sets  of  four  attention  allocation  ratings  were 
collected.  The  strategy  for  controlling  the  error  is  based  on  a  judicious  selection  of 
the  measures  for  analysis.  Further,  careful  consideration  is  given  to  the  statistics 
involved  since  inflated  Type  I  error  can  result  in  an  analysis  without  corrections 
for  departures  from  normality  in  the  within-subjects  effects.  Note  that  lack  of 
sphericity  is  not  a  problem  for  the  main  factors  in  this  analysis  since  only  two 
treatments  are  used  for  each  factor.  The  Holm  simultaneous  testing  procedure 
(Neter,  Kutner,  Nachtsheim,  &  Wasserman,  1996)  is  used  to  partition  the  overall 
alpha  level  of  .05  among  the  family  of  planned  separate  analyses.  The  Holm 
procedure  is  a  more  powerful  form  of  the  Bonferroni  procedure,  which  is  used  to 
control  family-wise  Type  I  error.  The  approach  is  now  described  in  greater  detail. 

The  overall  task  performance  of  target  detection  and  identification  is  determined 
with  a  canonical  correlation  analysis  (Pedhazur,  1982),  which  was  applied  to  the 
detection  and  identification  times  and  event  measures.  The  analysis  computes 
linear  combinations  for  the  dependent  and  independent  variables  so  that  the 
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maximum  possible  correlation  is  obtained  between  two  sets  of  canonical 
variates.  The  canonical  correlation  analysis  is  an  overall  statistical  test  and  does 
not  indicate  the  separate  contributions  of  the  multiple  dependent  variables  to  the 
differences  between  the  factorial  groupings.  As  an  additional  control  of  the  Type 
I  errors  following  overall  significance,  separate  univariate  analyses  are  applied  to 
the  dependent  measures  via  a  repeated  measure  (RM)  analysis  of  variance 
(ANOVA).  In  a  parametric  analysis,  the  distributions  for  the  dependent  measure 
must  satisfy  normality  and  homogeneity  of  variances  without  correlation  of  the 
variances  with  the  means.  To  maintain  the  error  rate  for  all  pair-wise 
comparisons,  the  contrast  of  post  hoc  comparisons  between  treatment  means  are 
computed  with  Tukey's  Honestly  Significant  Difference  Test  from  the  mean 
differences  and  the  ANOVA  error  residual  (Keppel,  1982).  Here,  sphericity  for 
the  within-subjects  effects  is  not  required  for  the  main  factors  since  there  are  two 
treatments  for  each  factor.  In  those  cases  in  which  the  distributions  for  the 
dependent  measure  are  not  normal,  a  nonparametric  (NP)  Friedman  ANOVA  by 
ranks  is  used  for  univariate  analysis,  followed  by  a  conservative  Scheffe  pair¬ 
wise  comparison  test  for  the  contrast  of  rank  means. 

Several  techniques  are  used  to  reduce  the  data  for  analysis  from  the 
questionnaires.  Different  researchers  have  developed  these  questionnaires,  and 
we  treat  the  results  in  separate  analyses.  First,  factor  analysis  (Cooley  &  Lohnes, 
1971)  is  used  to  reduce  the  measures  to  factorial  components  for  statistical 
analysis.  The  number  of  factorial  components  used  in  the  analysis  equals  the 
number  of  groupings  that  are  recognized  in  the  literature  as  being  appropriate  for 
the  questionnaire.  For  those  cases  in  which  no  such  standard  exits,  only  two 
factorial  components  are  used.  A  univariate  statistical  analysis  is  applied  to 
each  factorial  component  in  turn,  starting  with  the  first  factorial  since  this 
component  contains  the  largest  portion  of  the  variance  until  significance  is 
attained.  The  components  are  analyzed  by  parametric  methods  unless  the 
components  are  non-normal  distributions,  in  which  case,  a  nonparametric 
method  of  analysis  is  applied  to  the  first  component.  In  this  way,  the  fewest 
possible  analyses  are  applied  to  each  factorial  decomposition.  Following 
significance  of  the  factorial  component,  the  data  for  the  measures  are  analyzed 
according  to  standard  practice  for  the  questionnaire. 

It  is  standard  practice  to  reduce  the  data  for  the  perceived  performance 
questionnaires  (workload,  SA,  and  motion  sickness)  to  single  measures  by  a 
weighted  summing  of  the  component  scales.  For  example,  the  NASA-TLX 
workload  questionnaire  (Hart  &  Staveland,  1988)  consists  of  six  bi-polar  scales 
with  semantic  anchors,  three  scales  for  task  demand,  and  three  for  task 
interaction.  A  single  grand  measure  of  workload  may  be  calculated  by  summing 
the  ratings  of  the  component  scales  (Hendy,  Hamilton,  &  Landry,  1993).  Also  of 
interest  are  the  separate  sums  for  the  task  demands  and  interaction  scales.  In 
contrast,  the  SART  questionnaire  consists  of  ten  bi-polar  scales,  three  scales  for 
situational  demands,  four  for  supply,  and  three  for  situational  understanding.  A 
single  overall  measure  of  SA  is  calculated  from  the  sum  of  the  ratings  for  the 
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supply  and  understanding,  minus  the  sum  of  the  ratings  for  the  demand  scales 
(Selcon,  Taylor  &  Koritsas,  1991;  Taylor,  1988,  1989).  Of  further  interest  are  the 
separate  sums  for  the  demand,  supply,  and  understanding  ratings.  Finally,  the 
motion  sickness  questionnaire  (Kennedy  et  al.,  1989)  consists  of  16  multiple 
choices.  In  data  reduction,  these  are  mapped  to  numerical  scales  that  are  grouped 
into  ratings  for  a  nausea  symptom,  a  disorientation  symptom,  and  an  oculomotor 
symptom  (Kennedy  et  al.,  1992).  A  single  measure  of  total  severity  is  calculated 
from  the  weighted  sum  of  these  ratings. 

Once  the  planned  analyses  are  completed,  the  overall  family-wise  alpha  level  of 
.05  is  partitioned  among  the  statistical  tests  with  the  Holm  simultaneous  testing 
procedure  (Neter,  Kutner,  Nachtsheim,  &  Wasserman,  1996)  to  control  the  Type  I 
error.  With  this  procedure,  the  alpha  level  is  partitioned  among  the  family  of 
tests,  according  to  the  ranking  of  the  (two-sided)  probabilities  of  significance. 
With  each  acceptance  of  the  alternate  hypothesis,  the  family-wise  alpha  level  is 
adjusted  for  the  tests  remaining.  This  process  is  continued  until  a  test  is  reached 
for  which  the  null  hypothesis  applies;  the  null  hypothesis  is  then  accepted  for 
this  and  all  remaining  trials. 

3.8  Procedures 

The  experiment  was  conducted  in  May  1998,  with  the  trials  limited  to  the  hours 
between  1000  and  1400  to  provide  relatively  consistent  sky  lighting.  In  this 
experiment,  the  participants  searched  for  a  target  across  the  right  side  visual 
front  from  either  a  stationary  or  a  moving  HMMWV.  The  stationary  vehicle  was 
parked  to  the  left  side  of  the  course  at  an  observation  point  on  the  100-meter 
course  line.  The  moving  vehicle  was  driven  at  a  constant  10  miles  per  hour  along 
the  access  road  on  the  left  side  of  the  course  from  the  25-meter  line  toward  the 
550-meter  end.  The  participants  started  looking  for  a  target  when  the  moving 
vehicle  reached  the  50-meter  line  and  ended  their  search  at  the  150-meter  line;  in 
this  way,  the  participants  saw  practically  the  same  target  configurations  in  both 
vehicle  modes.  For  safety  reasons,  vehicle  movement  was  restricted  to  the  access 
road  on  the  left  side  of  the  firing  facility. 

The  targets  were  raised  for  display  and  lowered  one  at  a  time  under  computer 
control  by  the  SASPRF  personnel  supporting  the  experiment.  In  this  experiment, 
34  targets  are  used  for  training  and  investigation.  Table  5  lists  the  targets  with 
location  and  Landolt  ring  gap  orientation.  The  targets  are  plotted  in  Figure  9. 
They  are  positioned  about  the  150-meter  line,  200-meter  line,  250-meter  line,  and 
300-meter  line  of  the  course.  These  targets  extend  50  meters  to  225  meters  across 
a  60-degree  sector  to  the  right  side  from  the  100-meter  line  observation  point.  All 
but  the  six  far  right  targets  are  seen  against  a  light  green  field  grass  background; 
the  remaining  six  are  set  against  a  dark  brown  tree  line  background.  All  targets 
face  up  range  except  for  the  three  right  most  targets  which  were  turned  30 
degrees  to  the  right  of  centerline.  The  target  design  follows  from  a  preliminary 
study  which  showed  that  for  statistical  variations  in  sightings  to  occur,  the 
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targets  need  to  extend  from  50  meters  to  125  meters  for  the  indirect  vision  with 
the  HMD  and  200  meters  for  direct  vision  (see  Appendices  B  and  C). 


Table  5.  Target  Location  (meters)  and  Landolt  Ring  Gap  Orientation 


Target  x  y  Orientation  target  x  y  Orientation  target  x  y  Orientation 


A20 

96 

155 

R 

A21 

81 

150 

D 

A22 

68 

145 

R 

A23 

96 

205 

U 

A24 

81 

200 

D 

A25 

68 

195 

L 

A26 

96 

255 

R 

A27 

81 

250 

U 

A28 

68 

245 

R 

A29 

96 

305 

L 

A30 

81 

300 

R 

A31 

68 

295 

R 

B20 

40 

155 

D 

B21 

27 

150 

L 

B22 

13 

145 

L 

B23 

40 

205 

R 

B24 

27 

200 

U 

B25 

13 

195 

D 

B26 

40 

255 

R 

B27 

27 

250 

U 

B28 

13 

245 

L 

B29 

40 

305 

D 

B30 

27 

300 

U 

B31 

13 

295 

D 

C20 

-13 

155 

Uab 

C23 

-13 

205 

L 

C24 

-27 

200 

D 

C25 

-41 

195 

La 

C26 

-13 

255 

R 

C27 

-27 

250 

U 

C28 

-41 

245 

Da 

D23 

-69 

205 

L* 

D24 

-82 

200 

Dab 

D25 

-96 

195 

I_Tb 

"tree  line  background 
'’angular  offset 

Note:  Coordinate  system  origin  at  center  of  firing  line  with  positive  y-axis  down  range  and 
positive  x-axis  to  left  of  centerline. 

Gap  orientation  Key:  L-left,  R-right,  U-up,  D-down 


Figure  9.  Target  Deployment. 
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Eight  sets  of  target  sequences  were  used  in  the  experiment.  The  sequences  were 
drawn  from  target  groups  formed  by  subdividing  the  SASPRF  into  three  equal 
50-meter  bands  and  the  sector  into  three  equal  20-degree  zones.  Eight  sets  of  17 
targets  with  5  for  training  and  12  for  evaluation  were  selected  so  that  all  bands 
and  sector  zones  were  equally  represented  as  closely  as  possible.  The 
presentation  order  of  the  targets  within  each  set  was  randomly  selected  so  that 
all  ranges  and  bearings  appeared  equally  likely.  See  Table  6  for  the  order  of 
target  presentations.  All  participants  saw  the  target  sequence  sets  in  the  same 
order  because  of  the  limited  number  of  targets  available  and  to  facilitate  control. 


Table  6.  Target  Presentation  Order 


Target  Sets 

Trial  Phase 

Order 

1 

2 

3 

4 

5 

6 

7 

8 

Training 

1. 

B22 

A28 

C23 

B26 

A26 

B28 

B25 

A25 

2. 

B26 

B23 

A20 

B20 

B22 

A23 

B20 

C23 

3. 

A23 

A20 

A28 

A22 

A21 

B20 

A22 

B22 

4. 

A27 

C23 

B20 

A26 

B23 

C25 

A28 

B26 

5. 

A22 

B20 

B25 

B24 

C25 

A20 

C23 

B25 

Evaluation 

6. 

C23 

B22 

B30 

C27 

B26 

B29 

A20 

B23 

7. 

Nnn 

B25 

A21 

Nnn 

B20 

Nnn 

C27 

Nnn 

8. 

B20 

C28 

C25 

A28 

D25 

A26 

A27 

D24 

9. 

C26 

C20 

B24 

C20 

Nnn 

Nnn 

B26 

A29 

10. 

A20 

Nnn 

B26 

Nnn 

A31 

B21 

A25 

B20 

11. 

Nnn 

A29 

A23 

C25 

C26 

B25 

C24 

Nnn 

12. 

A25 

C24 

Nnn 

Nnn 

C24 

Nnn 

D25 

B28 

13. 

D24 

A27 

C27 

A29 

Nnn 

C23 

Nnn 

A23 

14. 

A30 

A24 

Nnn 

Nnn 

A20 

A24 

B31 

Nnn 

15. 

B28 

Nnn 

B21 

D24 

B31 

Nnn 

Nnn 

A21 

16. 

B23 

D25 

Nnn 

B23 

A27 

B27 

C20 

Nnn 

17. 

A28 

Nnn 

A31 

B28 

Nnn 

D23 

Nnn 

C26 

18. 

Nnn 

A22 

Nnn 

B22 

C20 

C28 

B22 

C20 

19. 

C20 

Nnn 

A26 

A21 

Nnn 

A22 

A31 

A28 

20. 

Nnn 

B27 

C20 

B30 

A23 

A30 

Nnn 

B30 

21. 

B29 

B31 

D23 

A25 

B24 

C20 

B24 

C25 

Nnn  =  no  target  presented. 


The  participants  were  evaluated  in  a  counterbalanced  manner  according  to  the 
schedule  of  Table  7,  which  shows  that  the  trial  runs  were  grouped  in  blocks  by 
the  vehicle  motion,  the  vision  mode,  and  the  cueing  format  for  experimental 
convenience.  That  is,  the  participant  was  evaluated  first  on  all  runs  for  the 
vehicle  in  motion,  followed  by  those  runs  for  the  parked  vehicle  or  in  the  reverse 
order.  Further,  the  participants  were  evaluated  in  a  counterbalanced  manner 
with  the  direct  or  indirect  vision  mode  for  each  vehicle  motion  block.  For  each 
vision  mode,  they  received  a  block  of  experimental  runs  without  localizing  cues 
and  a  block  with  audio  cueing.  For  each  such  block  combination,  the  participant 
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was  first  trained  in  that  mode  with  the  first  five  targets  of  the  scheduled  target 
set  sequence.  He  was  then  evaluated  on  a  sequence  of  16  presentations  consisting 
of  the  remaining  12  targets  from  that  sequence  set  and  4  additional  no-target 
presentations  randomly  interspersed  as  a  check  on  the  false  alarm  response. 


Table  7.  Participants'  Assignment 


Vehicle 

Stationary 

Moving 

Mode 

Direct 

HMD 

Direct 

HMD 

Cueing 

None 

3Dsd 

None 

3Dsd 

None 

3Dsd 

None 

3Dsd 

SI 

1 

2 

3 

4 

5 

6 

7 

8 

S2 

5 

6 

7 

8 

1 

2 

3 

4 

S3 

3 

4 

1 

2 

7 

8 

5 

6 

S4 

7 

8 

5 

6 

3 

4 

1 

2 

S5 

2 

1 

4 

3 

6 

5 

8 

7 

S6 

6 

5 

8 

7 

2 

1 

4 

3 

S7 

4 

3 

2 

1 

8 

7 

6 

5 

S8 

8 

7 

6 

5 

4 

3 

2 

1 

3Dsd  =  three-dimensional  sound 


3.9  Participant's  Scenario 

Following  the  participant's  arrival  at  the  facility,  the  experiment  was  described  to 
him.  A  volunteer  agreement  affidavit  (see  Appendix  A)  was  given  to  him  to  read, 
and  informed  consent  was  explained. 

The  participant  received  an  orientation  and  safety  briefing  after  he  agreed  to  take 
part  in  the  experiment  and  signed  the  volunteer  agreement.  The  visual  acuity 
(far)  and  self-reported  hearing  acuity  (left  and  right  ears)  data  were  collected 
from  the  participant,  along  with  demographic  data.  The  procedure,  the 
participant's  role  as  an  experimental  subject,  and  the  purpose  and  nature  of  the 
questionnaires  were  explained  to  him.  After  these  preliminaries,  the  participant 
was  assigned  a  sequence  of  vehicle  motion,  vision  mode,  and  cueing  format 
blocking  combinations,  in  accordance  with  the  schedule. 

Before  the  study  was  started,  the  participant  viewed  all  targets  in  the  upright 
position  from  the  100-meter  observation  point.  All  participants  were 
knowledgeable  of  the  facility  since  they  had  fired  small  arms  for  qualification  at 
previous  times.  They  quickly  learned  which  targets  were  used  in  the  study,  and 
by  their  viewing  the  targets,  this  learning  effect  was  removed  from  the  data. 
Otherwise,  the  learning  effect  would  have  increased  the  statistical  variance  and 
thus  reduced  the  significance. 

The  participant  was  trained  and  evaluated  on  the  target  sequence  set  for  each 
block  combination  as  determined  by  the  schedule.  Before  the  experimental 
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sequence,  the  HMD  was  put  on  the  safety  helmet  or  removed  as  needed  for  the 
block  combination.  The  light  shielding  was  put  in  place  about  the  passenger's 
area  or  removed  with  the  right  windshield  and  forward  door,  accordingly.  The 
participant  placed  the  earphones  and  donned  the  safety  helmet.  For  those 
sessions  that  involved  audio  localization,  the  participant  set  the  sound  amplitude 
and  checked  the  sound  localization  with  a  demonstration  computer  program 
before  experimentation  began.  The  experimenter  set  the  data  files  for  the  on¬ 
board  computer  and  activated  the  process  control  program.  During  the  initial 
training  for  those  sessions  with  sound  localization,  the  experimenter  checked 
with  the  participant  to  ensure  that  the  stereophonic  sound  cueing  appeared  to 
originate  from  the  targets. 

At  the  start  of  a  run,  all  targets  were  down,  and  the  participant  turned  toward 
the  facility  front  with  his  eyes  closed  and  his  head  down.  In  the  indirect  vision 
mode,  the  participant  aligned  the  scene  in  the  HMD  with  the  marker  and  closed 
his  eyes  while  the  experimenter  viewed  the  camera  return  on  a  separate  video 
monitor.  The  experimenter  alerted  facility  control  to  activate  the  target  by 
quickly  raising  a  white  paddle.  A  second  later,  the  experimenter  told  the 
participant  to  commence  searching  for  the  raised  target.  In  the  sound  localization 
mode,  the  experimenter  activated  the  sound  cues  as  well.  The  1-second  delay 
allowed  the  controller  to  respond  and  the  target  mechanism  to  activate.  The 
participant  informed  the  experimenter  by  voice  when  he  had  spotted  the  target 
and  the  experimenter  confirmed  the  sighting  to  the  facility  controller  by  again 
raising  his  white  paddle.  Using  the  electronic  telescope  at  full  magnification  in 
indirect  vision  or  a  set  of  hand-held  binoculars  in  direct  vision,  the  participant 
then  zoomed  in  on  the  target  and  informed  the  experimenter  of  the  Landolt 
ring's  orientation.  Again,  the  experimenter  quickly  raised  his  paddle  and  told  the 
controller  of  the  reported  orientation  with  a  hand  radio  set.  The  experimenter 
was  seated  behind  the  participant  and  the  paddle  was  beyond  his  field  of  vision. 
The  participant  was  allowed  no  more  than  25  seconds  to  complete  the  trial, 
15  seconds  to  detect,  and  10  seconds  after  detection  to  identify  the  target.  The 
facility  control  officer  pressed  a  computer  keypad  button  when  the  paddle  was 
raised  for  detection  and  identification  and  entered  the  reported  orientation.  The 
facility  control  computer  recorded  the  target  designator,  the  time  that  it  was 
raised,  and  the  times  that  the  button  was  pushed  for  detection  and  identification, 
along  with  the  orientation.  The  experimenter  was  in  radio  contact  with  the 
facility  control  personnel  at  all  times  through  a  hand  radio  set.  Facility  control 
told  the  experimenter  when  the  trial  time  elapsed  if  a  detection  or  identification 
was  not  made.  In  the  sound  localization  mode,  the  experimenter  turned  off  the 
sound  cues  at  the  end  of  the  run. 

During  a  run  with  direct  vision,  the  participant  sat  in  the  passenger's  seat  of  the 
vehicle  and  wore  the  safety  helmet  with  earphones.  The  right  windshield  and  the 
front  right  door  had  been  removed  to  allow  an  unobstructed  view  to  the  right 
front  and  side.  The  participant  searched  with  natural  unaided  vision  for  the 
target  and  then  used  a  set  of  hand-held  binoculars  to  see  the  Landolt  ring's 
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orientation.  In  indirect  vision,  the  participant  sat  in  the  front  passenger's  seat  and 
wore  the  safety  helmet  with  the  HMD  and  earphones.  The  front  passenger's  area 
had  been  completely  enclosed  with  light  shielding.  The  participant  saw  the  target 
through  the  HMD  with  a  video  link  to  the  head-coupled  electronic  camera  on  top 
of  the  vehicle.  As  the  participant  turned  his  head,  the  camera  turned  with  him. 
The  participant  searched  with  the  camera  at  full  FOV  for  the  target  and  then  used 
the  telescopic  zoom  to  see  the  Landolt  ring's  orientation.  He  manually  controlled 
the  FOV  of  the  camera  with  a  zoom  control  that  he  held  in  his  lap  during  the 
experiment.  In  sound  localization,  the  participant  heard  a  computer-controlled 
beeping  sound  in  his  ears,  apparently  coming  from  the  direction  of  the  target. 
While  in  the  stationary  vehicle,  the  participant  searched  from  the  100-meter  line. 
In  the  moving  vehicle,  he  started  his  search  as  the  vehicle  passed  the  50-meter 
line  and  ended  at  the  150-meter  line.  At  10  mph,  the  vehicle  reached  the  117- 
meter  line  in  about  15  seconds  and  the  150-meter  line  10  seconds  later.  Of  course, 
the  vehicle  was  moving  toward  the  target,  and  the  actual  detection  and 
identification  distances  were  a  function  of  the  search  times.  At  the  end  of  the  rim, 
the  vehicle  was  stopped  and  backed  to  the  start  of  the  course  at  the  25-meter 
mark  in  preparation  for  the  next  run. 


At  the  end  of  each  target  set  sequence,  the  participant's  heart  rate  was  recorded. 
Heart  rate  has  been  reported  to  be  an  indicator  of  workload,  and  the  participants 
in  this  experiment  were  in  a  sitting  position  with  physical  activity  limited  to  head 
movements  and  speech.  During  the  10-minute  rest  period  between  sessions,  the 
participants  answered  questionnaires  and  a  workload  battery  while  the 
ecfu*Pment was  being  prepared  for  the  next  session. 

At  the  end  of  the  experiment,  the  participants  were  debriefed  about  the 
performance  of  the  indirect  vision  system  and  the  sound  localization  in  the 
stationary  and  moving  vehicle,  and  any  questions  were  answered.  In  addition, 
the  participants  answered  a  questionnaire  for  the  subjective  evaluation  of  the 
systems  by  the  perceived  performance. 


4.  Statistical  Results 


The  results  of  the  statistical  analyses  of  the  target  detection  and  identification 
times,  numbers,  and  errors,  along  with  the  analyses  of  the  heart  rate  and  the 
ratings  from  the  questionnaires,  are  reported  here. 

4.1  Task  Performance  for  Detection  and  Identification 

The  overall  task  performance  was  determined  with  a  canonical  correlation 
analysis  (Pedhazur,  1982)  applied  to  the  detection  and  identification  times  and 
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event  measures,  using  the  multivariate  fitting  option  of  the  SAS3  JMP3  statistical 
software  package.  The  dependent  time  measures  used  in  the  analysis  are  the 
natural  logarithmic  transformations  of  the  times  to  detect  and  the  times  to 
identify  after  detection.  The  time  to  complete  the  trial,  which  was  limited  to  15 
seconds,  was  used  as  the  detection  time  for  those  presentations  when  a  target 
was  not  detected;  10  seconds  were  used  for  an  identification  time  for  failures  to 
identify  after  detection.  The  detection,  identification,  and  correct  identification 
events  were  "dummy"  coded  as  categorical  variables,  with  a  success  assigned  the 
value  of  unity  and  a  failure,  the  value  0.  There  were  no  false  alarm  detections 
since  none  of  the  participants  reported  targets  on  any  of  the  runs  when  no  targets 
were  presented.  The  fixed  factors  of  the  experiment  were  represented  as 
categorical  variables  by  effect  coding  and  the  factorial  interactions  by  the  coding 
products.  The  target  distance  and  azimuth  are  continuous  variables  computed 
for  the  time  of  detection.  Since  the  experiment  is  within  subjects  with  repeated 
measures,  the  subjects  were  represented  as  categorical  variables  by  their  total 
times  to  detect  as  a  criterion  scale  coding.  In  turn,  since  criterion  scaling  is  used 
in  place  of  effect  coding,  the  degrees  of  freedom  in  the  analysis  are  corrected  to 
account  for  the  number  of  subjects. 

The  canonical  correlation  analysis  is  an  overall  statistical  test  and  does  not 
indicate  the  separate  contributions  of  the  multiple  dependent  variables  to  the 
differences  between  the  factorial  groupings.  The  structure  coefficients  or 
loadings  derived  from  the  canonical  weights  are  measures  of  the  correlations 
between  the  variables  and  the  canonical  variate  scores.  For  this  reason,  the 
logarithmic  time  variables  were  analyzed  with  separate  parametric  multiple 
linear  regressions  (Pedhazur,  1982),  with  the  Statistical  Package  for  the  Social 
Sciences  (SPSS)  8.0.  The  independent  variables  are  the  same  as  for  the  canonical 
correlation.  Similarly,  the  detection,  identification,  and  correct  identification 
events  were  analyzed  in  separate  discriminant  analyses  (Pedhazur,  1982;  Cooley 
&  Lohnes,  1971),  in  which  the  success  and  failure  of  the  event  are  interpreted  as 
members  of  separate  populations.  Here  the  dependent  variable  of  membership  is 
categorical  and  nominal  in  scaling. 

The  validity  of  the  multiple  regression  analyses  depends  on  the  satisfaction  of 
certain  requirements  for  the  error  residuals.  The  conditions  on  error  distributions 
(normal  with  zero  means)  and  variances  (homoscedasticity  or  constant  for  all 
groupings)  are  satisfied  when  the  frequency  distributions  for  the  dependent  and 
independent  variables  are  symmetrical.  The  detection  and  identification  time 
distributions  are  skewed  toward  the  lower  values  while  being  limited  by  the 
occurrences  of  elapsed  time.  However,  the  natural  logarithmic  transformations  of 
the  detection  and  identification  times  are  close  to  normal  and  for  this  reason,  are 
analyzed.  The  categorical  codings  are  symmetrical  as  fixed  factors.  The 
continuous  variable  distributions  of  distance  and  azimuth  are  close  to  normal  by 
design.  While  the  fixed  factors  were  blocked  in  presentation,  the  targets  were 
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presented  in  a  random  order  by  distance  and  azimuth  resulting  in  serial 
independence  of  the  error  residuals  and  lack  of  correlation  with  the  independent 
variables. 

4.1.1  Overall  Task  Performance 

The  results  of  the  canonical  correlation  analysis  shows  that  the  measures  of  task 
performance  are  significantly  influenced  by  the  vehicle  movement  (p  <  .0001) 
and  vision  (p  <  .0001)  modes,  and  the  interaction  between  the  movement  and 
vision  (p  <  .0001).  The  whole  model  Wilks'  lambda  value  of  0.1778  with  a 
maximum  canonical  correlation  of  84.05%,  corresponds  to  a  p  <  0.0001  level  of 
statistical  significance  (approx  F  =  18.192,  dfn  =  85,  dfe  =  3601). 

4.1.2  Detection  Times 

The  natural  logarithms  of  the  detection  times  (including  the  15-second  time 
allotted  to  the  incomplete  trials)  are  significantly  influenced  by  the  vehicle 
movement  (p  <  .001)  and  vision  (p  <  .001)  treatments  and  by  the  three-way 
interaction  among  the  vehicle  movement,  vision,  and  audio  cueing  treatments 
(p  <  .004).  The  overall  model  ANOVA  is  statistically  significant  (F  =  87.676, 
dfn  — 17,  dfe  =  748,  p  <  .0005),  indicating  that  some  model  coefficients  are 
significantly  different  from  zero.  The  regression  model  with  a  81.6%  correlation 
coefficient,  accounts  for  66.6%  of  the  variance;  however,  since  this  is  an 
overestimation  of  the  population  correlation  (r-square),  the  adjustment  for  the 
number  of  model  variables  and  sample  size  is  66.1%.  The  standard  error  of  the 
estimate  (square  root  of  the  residual  mean  square)  is  0.4484.  As  with  any 
multiple  regression  analysis,  the  t-statistics  for  the  model  coefficients,  obtained 
by  dividing  the  coefficient  by  its  standard  error,  is  a  measure  of  the  statistical 
significance  of  the  coefficients  with  the  standardized  "Beta"  coefficients  for  the 
data  transformed  to  z-scores,  which  are  used  to  compare  the  coefficients. 

Figure  10  is  a  plot  of  the  average  detection  times  with  95%  confidence  intervals 
for  the  treatment  conditions  of  vehicle  movement,  vision  mode,  and  audio 
cueing  format.  The  figure  shows  the  main  effects  and  the  interactions  among  the 
three  modes.  The  results  are  that  on  the  average,  the  targets  were  detected 
quickest  with  direct  viewing  from  the  stationary  vehicle  without  audio 
localization.  On  the  average,  the  targets  were  detected  sooner  with  direct 
viewing  than  with  the  F1MD,  and  for  the  FIMD,  sooner  from  the  stationary 
position  than  the  moving  vehicle. 

A  study  of  the  confidence  intervals  plotted  in  Figure  10  suggests  that  for  the 
HMD,  there  are  insignificant  differences  between  the  cued  and  non-cued  times 
for  the  moving  and  stationary  modes.  In  addition,  there  are  insignificant 
differences  for  the  direct  vision  among  the  moving  cued  and  non-cued  modes 
and  the  cued  stationary.  The  results  suggest  that  the  three-way  interaction  is 
attributable  to  the  significant  difference  of  the  non-cued,  stationary  mode  for  the 
direct  vision  from  the  rest  of  the  treatments.  An  application  of  Scheffe's  post  hoc 
two-way  comparison  test  shows  that  the  detection  times  (natural  logarithms)  for 
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this  treatment  mode  are  significantly  less  (Scheffe's  F  value  =  107.013,  df  =  1,  dfe 
=  748,  p  <  .05)  than  those  for  the  remaining  modes.  A  study  of  the  data  shows  no 
outliers  as  a  source  of  this  difference. 


Figure  10.  Average  Detection  Times  (including  elapsed  time)  With  95% 
Confidence  Intervals. 

4.1.3  Detection  Events 

The  detection  events  are  significantly  influenced  by  the  vehicle  movement 
(p  <  .001)  and  vision  (p  <  .001)  modes  and  their  interaction  (p  <  .001).  The 
discriminant  model  with  a  86.7%  correlation  coefficient,  accounts  for  75.3%  of  the 
variance  (adjusted  r-square  is  71.7%;  standard  error  of  the  estimate  is  0.01515). 
The  F  statistic  for  the  overall  model  ANOVA  is  significant  (F  =  10.642,  dfn  =  14, 
dfe  =  49,  p  <  .001). 

Figure  11  is  a  plot  of  the  number  of  detections  as  a  function  of  the  conditions  of 
vehicle  movement,  vision  modes,  and  cueing  format.  The  results  are  that  more 
targets  were  detected  with  direct  viewing  than  with  the  HMD  and  for  the  F3MD 
from  the  stationary  position  than  the  moving  vehicle.  An  application  of  the 
Scheffe's  post  hoc  two-way  comparison  test  shows  that  the  number  of  detections 
for  the  moving  vehicle  with  the  HMD  are  significantly  less  (Scheffe's 
F-value  =  12.972,  dfn  =  1,  dfd  =  49,  p  <  .05)  than  those  for  the  remaining  modes. 
Although  there  is  a  trend  for  more  targets  being  detected  with  direct  viewing 
from  the  stationary  vehicle  without  audio  cueing,  the  audio  cueing  increased  the 
number  of  targets  detected  in  all  other  treatments. 
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Figure  11.  Total  Number  of  Targets  Detected  With  95%  Confidence  Intervals. 


4.1.4  Identification  Times 

The  identification  times  (including  the  10-second  time  allotted  to  the  incomplete 
trials)  are  significantly  influenced  by  the  vehicle  movement  (p  <  .001)  and  vision 
(p  <  .001)  modes  and  their  interaction  {p  <  .001).  The  regression  model  with  a 
68.2%  correlation  coefficient,  accounts  for  46.5%  of  the  variance  (adjusted 
r-square  is  45.7%).  The  standard  error  of  the  estimate  is  0.3626.  The  F  statistic  for 
the  overall  model  ANOVA  is  significant  (F  =  38.379,  dfn  =  17,  dfe  =  748,  p  <  .001). 

Figure  12  is  a  plot  of  the  average  identification  times  as  a  function  of  the 
treatment  conditions  of  vehicle  movement,  vision  modes,  and  cueing  format. 
Comparing  all  treatments,  the  fastest  identifications  were  made  with  direct 
viewing  from  the  stationary  vehicle.  Considering  the  vision  modes,  the  targets 
were  identified  faster  with  direct  viewing  than  with  the  HMD.  An  application  of 
Scheffe's  post  hoc  two-way  comparison  test  shows  that  the  identification  times 
(natural  logarithms)  for  the  moving  vehicle  with  the  HMD  are  significantly 
greater  (Scheffe's  F-value  =  22.754,  dfn  =  1,  dfd  =  748,  p  <  .05)  than  those  for  the 
remaining  modes.  Audio  cueing  shows  a  trend  for  decreased  identification  times 
in  all  treatments  except  for  the  HMD  from  the  moving  vehicle. 
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Figure  12.  Average  Identification  Times  (including  elapsed  time)  With  95% 
Confidence  Intervals. 

4.1.5  Identification  Events 

The  identifications  are  significantly  influenced  by  the  vehicle  movement 
(p  <  .001)  and  vision  (p  <  .001)  modes,  and  less  so  by  their  interaction  (p  <  .013). 
The  discriminant  model  with  a  90.5%  correlation  coefficient  accounts  for  82.0%  of 
the  variance  (adjusted  r-square  is  79.3%);  the  standard  error  of  the  estimate  is 
0.0233.  The  F  statistic  for  the  overall  model  ANOVA  is  highly  significant 
(F  =  15.977,  dfn  =  14,  dfd  =  49,  p  <  .001).  Figure  13  is  a  plot  of  the  number  of 
identifications  as  a  function  of  the  treatment  conditions  of  vehicle  movement, 
vision,  and  cueing  mode.  The  figure  shows  the  main  effects  and  the  interactions 
among  the  experimental  conditions.  The  results  are  that  more  targets  were 
identified  with  direct  viewing  than  with  the  HMD  and  from  the  stationary 
position  than  the  moving  vehicle.  More  targets  were  identified  with  direct 
viewing  from  the  stationary  vehicle  than  in  the  other  conditions.  An  application 
of  Scheffe's  post  hoc  two-way  comparison  test  shows  that  on  the  number  of 
identifications  for  the  moving  vehicle  with  the  HMD  are  significantly  less 
(Scheffe's  F-value  =  19.594,  dfn  =  1,  dfd  =  49,  p  <  .05)  than  those  for  the  remaining 
modes. 

4.1.6  Correct  Identifications 

The  statistics  for  the  targets  correctly  identified  follow  closely  those  for  the 
identifications.  The  correct  identifications  are  significantly  influenced  by  the 
vehicle  movement  (p  <  .001)  and  vision  (p  <  .001)  modes.  The  discriminant  model 
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with  a  87.5%  correlation  coefficient  accounts  for  76.6%  of  the  variance  (adjusted 
r-square  is  73.2%);  the  standard  error  of  the  estimate  is  0.0244.  The  F  statistic  for 
the  overall  model  ANOVA  is  highly  significant  (F  =  11.513,  dfn  =  14,  dfd  =  49, 
V  <  -001).  The  numbers  of  correct  identifications  plotted  in  Figure  14  are 
significantly  influenced  by  the  vehicle  movement  and  vision  modes.  The  results 
are  that  more  targets  were  identified  correctly  with  direct  viewing  than  with  the 
HMD  and  from  the  stationary  position  than  the  moving  vehicle.  More  targets 
were  identified  correctly  with  direct  viewing  from  the  stationary  vehicle  than  in 
the  other  conditions. 


Figure  13.  Total  Number  of  Targets  Identified  Per  Treatment. 


A  study  of  the  error  residuals  for  the  regression  analyses  listed  shows  that  the 
requirements  for  parametric  analysis  are  met  in  all  cases.  The  residuals  for  the 
effect  groupings  appear  normally  distributed  with  zero  means  and  nearly  zero 
skewness  and  kurtosis,  satisfy  homoscedasticity  with  constant  variance,  are 
serially  independent,  and  are  not  correlated  to  the  independent  effects.  The 
histograms  and  normal  probability  plots  of  the  standardized  residuals  for  the 
dependent  variable  are  closely  normal  in  distribution.  Similarly,  the  partial 
regression  scatter  plots  of  the  residuals  plotted  against  the  independent  variables 
show  the  points  scattered  in  a  horizontal  band  with  no  apparent  systematic 
features,  as  would  be  expected  if  no  relations  remained  after  regression. 
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Figure  14.  Total  Number  of  Targets  Identified  Correctly  Per  Treatment. 

4.2  Heart  Rate 

Heart  rate  was  not  significantly  influenced  by  the  treatments,  as  determined  by  a 
univariate  RM  ANOVA.  The  average  heart  rate  experienced  by  the  participants 
was  79.6  beats  per  minute.  The  heart  rates  were  measured  at  the  end  of  the  trial 
runs  for  each  block. 

4.3  Subjective  Questionnaires 

The  scores  for  the  questionnaires  are  analyzed  for  (a)  the  attention  allocation 
loading  factors  for  the  detection  and  identification  tasks,  (b)  the  NASA-TLX  of 
perceived  workload,  (c)  Kennedy's  subjective  estimation  of  motion  sickness,  and 
(d)  Selcon  and  Taylor's  SART.  The  attention  allocation  loading  factors  are 
measures  of  sensory,  cognitive,  and  motor  channel  loading,  which  are  particular 
to  the  tasks  as  experienced  by  the  participants.  The  workload  battery,  motion 
sickness,  and  SA  ratings  are  measures  of  the  subjectively  perceived  states  by  the 
participants  as  influenced  by  the  task  and  environment. 

Although  the  overall  significance  for  each  questionnaire  is  determined  by  the 
statistical  analysis  of  the  factorial  components,  the  statistical  study  of  the  scores 
for  the  questionnaire  is  of  interest.  However,  a  study  of  the  rating  scores  for  the 
questionnaires  shows  that  in  many  cases,  the  data  for  the  treatment  effects  do  not 
satisfy  the  conditions  for  a  parametric  ANOVA.  For  many  questions,  the 
distributions  are  not  normal  or  the  variances  are  not  homogeneous.  This  is 
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especially  true  for  the  motion  sickness  questionnaire,  in  which  most  ratings  were 
assigned  a  zero  score  for  all  but  the  treatment  of  the  moving  vehicle  with  the 
HMD,  and  several  participants  assigned  a  value  of  zero  to  this  condition.  For  this 
reason,  the  statistical  significance  of  the  scores  is  analyzed  with  the  NP  Friedman 
AXOYA  by  ranks  for  matched  samples,  following  ranking  of  the  rating  scores. 

The  statistical  approach  used  to  analyze  the  score  data  is  different  for  each 
questionnaire.  The  scores  for  the  loading  factors  for  the  detection  and 
identification  tasks  are  analyzed  in  separate  statistical  tests  for  the  sensory, 
cognitive,  and  motor  channels.  The  factors  are  estimates  of  the  task  loading  on 
the  channels  as  separate  components  of  an  attention  resource  model  of  human 
performance. 


The  scores  for  the  NASA -TLX  workload  battery  are  analyzed  as  an  overall  sum, 
the  sums  for  the  demand  and  interaction,  and  the  component  scores.  This  is 
because  the  separate  components  represent  different  kinds  of  workload.  Here, 
the  mental,  physical,  and  temporal  dimensions  measure  the  demands  placed  on 
the  participant,  while  the  effort,  frustration,  and  performance  dimensions  relate 
to  the  interaction  with  the  task. 


Similar  comments  apply  to  the  analyses  of  the  motion  sickness  ratings.  An 
overall  estimation  of  motion  sickness  formed  from  the  sum  of  the  ratings  for  the 
components  is  statistically  analyzed  in  an  NP  ANOVA.  Further,  the  components 
represent  degrees  of  motion  sickness  extending  from  general  discomfort  to 
extreme  nausea  and  are  analyzed  in  separate  statistical  tests. 


Again,  similar  comments  apply  to  the  analysis  of  the  SART.  The  scores  for  the 
demands  upon  attentional  resources  and  those  for  the  support  (supply)  of 
attentional  resources  are  measures  of  the  workload  used  to  maintain  SA.  An 
overall  rating  (R)  is  obtained  by  the  difference  between  the  sum  of  the 
components  for  the  supply  (S)  and  the  demand  (D)  added  to  those  for 
understanding  of  the  situation  (U),  i.e.,  R  =  S-  D  +  U.  For  this  reason,  estimates  of 
the  overall  rating,  along  with  the  sums  of  the  demand,  support,  and 
understanding  scores  are  analyzed  separately,  as  are  the  scores  for  the  separate 
questions. 

4.3.1  Attentional  Allocation  Factors 

The  allocations  of  the  attentional  resources  to  the  auditory,  visual,  cognitive,  and 
psychomotor  processing  channels  are  significantly  different  for  the  treatments,  as 
estimated  by  the  attention  allocation  loading  factors.  This  is  true  of  the  auditory 
and  psychomotor  channels  for  the  detection  task,  and  the  auditory  channel  for 
the  identification  task. 
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4.3.1. 1  Detection  Task 

Figure  15  shows  a  factorial  component  loading  diagram  for  the  detection  task 
following  reduction  to  two  components  with  a  factor  analysis  using  principal 
component  analysis  as  the  extraction  method  (69.559%  total  variance  explained), 
and  Varimax  rotation  with  Kaiser  normalization.  Considering  the  loading 
diagram  to  represent  a  state  space  for  the  attention  loading,  the  first  factorial 
component  corresponds  to  visual  and  cognitive  processing,  while  the  second 
component  corresponds  to  motor  responses.  Here,  a  study  of  the  rotated 
component  weight  matrix  shows  that  the  first  component  is  dominated  by  the 
visual  (0.877,  0.114)  and  cognitive  (0.889,  0.102)  channels,  while  the  second 
component  is  dominated  by  the  psychomotor  channel  (-0.024,  0.846).  The 
auditory  channel  (0.226,  0.657)  has  elements  of  both  components. 
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Figure  15.  Factorial  Component  Plot  in  Rotated  Space  for  Allocation  of  Attention 
for  Detection. 
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Application  of  an  NP  RM  Friedman  ANOVA  test  by  ranks  shows  that  the  first 
factorial  component  varies  significantly  with  treatments  ( p  <  .009,  chi- 
square  =  18.792,  df  =  7,  N  =8).  Figure  16  is  an  exploratory  data  analysis  (EDA) 
"box-and-whisker"  plot  (Velleman  &  Payne,  1992)  of  the  first  component  for  each 
of  the  treatments.The  figure  shows  that  the  box  plots  for  the  stationary 
treatments  are  heavily  skewed  with  one  outlier  and  no  extreme  values.  For  an 
EDA,  the  boxplot  for  a  distribution  will  show  the  median,  the  "hinges"  (first  and 
third  quartiles),  and  the  maximum  and  minimum  values  that  are  not  outliers  or 
extreme  values.  Any  outlier  values  of  the  distribution  are  marked  by  an  open 
circle  (o)  and  the  extremes  are  marked  by  an  asterisk  (*).  Values  more  than  1.5 


31 


times  the  box  lengths  (interquartile  range)  from  the  quartiles  are  designated  as 
outliers  and  values  more  than  three  box  lengths  are  designated  as  extremes. 


Figure  16.  Box  Plots  for  the  First  Factorial  Component  of  the  Detection  Attention 
Allocations. 


As  a  confirmation  of  this  result,  the  separate  applications  of  the  univariate  NP 
RM  Friedman  ANOVA  test  by  ranks  shows  statistically  significant  differences 
among  the  treatments  for  the  psychomotor  channel  (chi-square  statistic  =  28.717, 
df  =  7,  p  <  0.001)  and  auditory  channel  (chi-square  statistic  =  49.800,  df  =  7, 
V  <  0.001).  A  Scheffe  pair-wise  comparison  test  for  the  contrast  of  rank  means 
shows  the  source  of  these  differences.  Significant  higher  attention  was  allocated 
to  the  psychomotor  channel  with  the  HMD  compared  to  that  with  direct  viewing 
(Scheffe  test  statistic  =  24.0,  df  =  7,  p  <  .01).  Significantly  higher  allocations 
occurred  for  the  auditory  channel  with  sound  localization  (Scheffe  test  statistic  = 
42.667,  df  =  7,  p  <  .001)  compared  to  that  with  no  cues. 

The  box  plots  of  Figures  D-l  through  D-4  in  Appendix  D  show  the  visual, 
cognitive,  auditory,  and  psychomotor  channel  loading  scores.  For  reference,  the 
figures  show  the  verbal  anchors  for  the  equivalent  behavioral  functions  that 
correspond  to  the  loading  factors  (see  questionnaire  in  Appendix  A).  The  figures 
show  that  the  increase  in  average  loading  with  the  HMD  compared  to  that  for 
direct  vision  is  generally  true  for  all  channels.  While  the  allocation  increase  is 
statistically  significant  for  the  psychomotor  channel,  the  plots  show  increasing 
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trends  with  the  HMD  for  all  remaining  channels.  For  the  cognitive  channel,  the 
effect  is  level  for  all  treatments  except  the  audio  cueing  in  the  moving  vehicle 
where  the  cognitive  loading  is  reduced  for  direct  vision  but  markedly  increased 
with  the  HMD.  Finally,  the  significant  increase  in  auditory  attention  caused  by 
the  audio  cueing  is  coupled  with  an  increasing  trend  with  the  HMD. 

4.3.1.2  Identification  Task 

Figure  17  shows  a  factorial  component  loading  diagram  for  the  allocations  to  the 
four  channels  for  the  identification  task  after  reduction  to  two  components  with  a 
factor  analysis  using  principal  component  analysis  as  the  extraction  method 
(69.113%  total  variance  explained),  and  Varimax  rotation  with  Kaiser 
normalization.  A  study  of  the  rotated  component  weight  matrix  shows  that  the 
visual  (0.878,  0.025)  and  cognitive  (0.809,  0.105)  channels  dominate  one 
component,  while  the  auditory  channel  (0.240,  0.845)  dominates  the  other 
component,  and  the  psychomotor  (0.466,  -0.595)  is  divided  between  both 
components. 


Component  1 

Figure  17.  Factorial  Component  Plot  in  Rotated  Space  for  Allocation  of  Attention 
for  Identification. 

Application  of  an  NP  RM  Friedman  ANOVA  test  by  ranks  shows  that  the  second 
factorial  component  varies  significantly  with  treatments  (chi-square  =  24.458, 
df  =  7,p  <  .001)  but  not  the  first.  Figure  18  shows  an  EDA  box  plot  of  the  second 
component  for  the  treatments.  As  a  confirmation  of  these  results,  the  separate 
applications  of  the  NP  RM  Friedman  ANOVA  test  by  ranks  shows  statistically 
significant  higher  loadings  for  the  auditory  channel  (chi-square  statistic  =  45.798, 
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df  =  7,  p  <  0.001)  with  sound  localization  (Scheffe  test  statistic  =  35.042,  df  =  7, 
V  <  .001).  The  box  plots  of  Figures  E-l  through  E-2  show  the  loading  scores  for 
the  visual,  cognitive,  auditory,  and  psychomotor  channels.  The  plots  show  that 
the  attention  loading  is  in  the  main  level  across  vision  modes.  The  significant 
increase  in  auditory  attention  is  caused  by  the  audio  cueing;  however,  there  is  a 
trend  increase  in  psychomotor  attention  with  the  HMD  compared  to  that  for 
direct  vision. 


Figure  18.  Box  Plots  for  the  Second  Factorial  Component  of  the  Identification 
Attention  Allocations. 

4.3.2  NASA  Task  Loading  Index  (TLX)  Workload  Battery 

As  determined  by  analysis  of  the  first  factorial  component,  the  perceived 
workload  is  significantly  increased  by  the  use  of  the  HMD.  Figure  19  shows  a 
factorial  component  loading  diagram  for  the  six  TLX  scales  after  reduction  to  two 
components  with  a  factor  analysis  using  principal  component  analysis  as  the 
extraction  method  (76.953%  total  variance  explained)  and  Varimax  rotation  with 
Kaiser  normalization.  As  noted  in  the  key  on  the  figure,  the  TLX  scales  are  shape 
coded  by  their  status  as  task  demand  or  interaction.  Considering  the  loading 
diagram  to  represent  a  state  space  for  the  perceived  workload,  the  first  factorial 
component  corresponds  to  task  demand  while  the  second  component 
corresponds  to  interaction.  Here,  a  study  of  the  rotated  component  weight  matrix 
shows  that  the  first  component  is  dominated  by  the  physical  demand  (0.825, 
0.041)  while  the  second  component  is  dominated  by  the  task  effort  (0.126,  0.929). 
The  mental  (0.704,  0.432)  and  temporal  (0.793,  0.387)  demands  and  task 
performance  (0.711,  0.525)  and  frustration  (0.516,  0.739)  have  elements  of  both 
components. 
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Figure  19.  Factorial  Component  Plot  in  Rotated  Space  for  TLXs. 


Figure  20  shows  EDA  box  plots  of  the  first  component  for  the  viewing 
treatments.  The  plots  show  the  distributions  to  be  fairly  normal  with  closely 
equal  variances.  A  univariate  RM  ANOVA  test  of  within-subjects  effects  is 
significant  by  visual  treatments  ( p  <  .001,  F  =  50.036,  df  =  1,  error  df  =  7); 
however,  the  other  treatments  and  interactions  are  not  significantly  different 
from  each  other. 


Figure  20.  TLX  First  Factorial  Component  Box  Plots. 
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The  total  workload  is  significantly  greater  for  the  HMD  than  for  direct  vision. 
Figure  21  shows  EDA  box  plots  of  the  sum  of  the  workload  scores  for  the 
viewing  treatments.  The  univariate  RM  ANOVA  test  of  within-subjects  effects  is 
significant  by  visual  treatments  (p  <  .001,  F  =  43.862,  df  =  1,  error  df  =  7); 
however,  the  other  treatments  and  interactions  are  not  significantly  different 
from  each  other.  Similar  comments  apply  to  the  demand  and  interaction  sums 
and  component  scores. 


Figure  21.  TLX  Workload  Sum  Box  Plots. 


The  sum  of  the  demand  workload  scores  is  significantly  greater  for  the  FJMD 
than  for  direct  vision.  Figure  22  shows  EDA  box  plots  of  the  demand  sum  for  the 
viewing  treatments.  Here,  the  univariate  RM  ANOVA  test  of  within-subjects 
effects  is  significant  by  visual  treatments  ( p  <  .001,  F  =  30.322,  df  =  1,  error  df  =  7); 
however,  the  other  treatments  and  interactions  are  not  significantly  different 
from  each  other.  Application  of  the  NP  RM  Friedman  ANOVA  by  ranks  shows 
this  significant  difference  for  the  demand  scores.  A  Scheffe  pair-wise  comparison 
test  for  the  contrast  of  rank  means  shows  that  these  differences  are  primarily 
attributable  to  the  visual  treatments.  This  is  true  for  the  mental  demand  (chi- 
squared  statistic  =  27.956,  df  =  7,  p  <  .001;  Scheffe's  test  value  =  25.523,  df  =  7, 
pc. 001),  physical  demand  (chi-squared  statistic  =  23.012,  df  =  7,  p  <  .002; 
Scheffe's  test  value  =  19.260,  df  =  7,  p  <  .01),  and  temporal  demand  (chi-squared 
statistic  =  25.604,  df  =  7,  p  <  .001;  Scheffe's  value  =  23.010,  df  =  7,  p  <  .01). 
Figures  F-l  through  F-3  are  box  plots  for  these  workload  components  and 
demonstrate  the  increase  in  workload  demand  with  the  HMD.  As  can  be  seen 
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Figure  22.  TLX  Demand  Sum  Box  Plots. 


The  sum  of  the  interaction  workload  scores  is  significantly  greater  for  the  HMD 
than  direct  vision.  Figure  23  shows  EDA  box  plots  of  the  interaction  sum  for  the 
viewing  treatments.  Again,  the  univariate  RM  ANOVA  test  of  within-subjects 
effects  is  significant  by  visual  treatments  (p  <  .001,  F  =  30.614,  df  =  1,  error  df  =  7); 
however,  the  other  treatments  and  interactions  are  not  significantly  different 
from  each  other.  Application  of  the  NP  RM  Friedman  ANOVA  by  ranks  shows 
that  this  is  true  for  the  interaction  scores:  effort  (chi-squared  statistic  =  44.767,  df 
=  7,  p  <  .001;  Scheffe's  test  value  =  37.500,  df  =  7,  p  <  .001),  performance  (chi- 
squared  statistic  =  28.744,  df  =  7,  p  <  .001;  Scheffe's  test  value  =  24.00,  df  =  7,  p  < 
.001),  and  frustration  level  (chi-squared  statistic  =  33.153,  df  =  7,  p  <  .001; 
Scheffe's  test  value  =  28.711,  df  =  7,  p  <  .001).  Figures  F-4  through  F-7  are  box 
plots  for  these  workload  components  and  demonstrate  the  increase  in  workload 
interaction  with  the  HMD.  As  can  be  seen  from  the  figures,  the  distributions  are 
far  from  normal  with  non-homogeneous  variances,  as  would  be  expected  from 
the  summation  data. 


Figure  23.  TLX  Task  Interaction  Sum  Box  Plots. 


4.3.3  Kennedy's  Subjective  Estimation  of  Motion  Sickness 
As  determined  by  analysis  of  the  first  factorial  component,  motion  sickness  is 
significantly  greater  for  the  HMD  while  participants  are  in  the  moving  vehicle. 
Figure  24  shows  a  factorial  component  loading  diagram  for  the  16  motion 
sickness  scales  after  reduction  to  three  components  with  a  factor  analysis  using 
principal  component  analysis  as  the  extraction  method  (78.240%  total  variance 
explained),  and  Varimax  rotation  with  Kaiser  normalization.  As  noted  in  the  key 
on  the  figure,  the  scales  are  shape  coded  by  their  status  as  nausea,  oculomotor,  or 
disorientation  symptoms  (Kennedy  et  al.,  1992);  those  scales  used  in  multiple 
symptoms  are  only  coded  for  one  symptom.  A  study  of  the  rotated  component 
matrix  for  the  factorial  plot  of  Figure  24  suggests  that  the  first  factorial 
component  reflects  dizziness  and  cognitive  impediment;  the  second  component 
reflects  physical  impediment,  and  the  third  component  reflects  ocolomotor 
difficulties.  Although  not  an  exact  mapping  to  the  disorientation,  nausea,  and 
oculomotor  symptoms  determined  by  Kennedy  et  al.  (1992),  the  interpretations 
are  similar. 

Figure  25  shows  EDA  box  plots  of  the  first  factorial  component  for  the 
treatments.  The  plots  show  the  distributions  to  be  practically  zero  for  all 
treatments  except  for  moving  with  the  HMD,  and  heavily  skewed  with  outliers 
for  these  conditions.  An  NP  RM  Friedman  ANOVA  test  by  ranks  is  significant 
for  the  component  across  treatments  (chi-square  =  27.392,  df  =  7,  N  =  8,  p  <  .001). 


38 


Component  values 


Figure  24.  Motion  Sickness  Factorial  Component  Plot  in  Rotated  Space. 


Figure  25.  Motion  Sickness  First  Factorial  Component  Box  Plots. 
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The  total  severity  motion  sickness  is  significantly  greater  for  the  moving  vehicle 
with  the  HMD.  Figure  26  shows  EDA  box  plots  of  the  total  severity  as  a  function 
of  the  treatment.  The  distributions  are  largely  skewed  toward  low  values  except 
for  the  treatments  of  moving  with  the  HMD.  An  NP  Friedman  ANOVA  test  by 
ranks  applied  to  the  total  severity  shows  significant  differences  across  treatments 
(chi-square  =  34.893,  df  =  7,  N  =  8,  p  <  .001).  A  Scheffe  pair-wise  comparison  test 
for  the  contrast  of  rank  means  shows  that  this  difference  is  attributable  to  the 
HMD  in  the  moving  vehicle  (Scheffe's  test  statistic  =  22.222,  df  =  7,  p  <  .01). 


Figure  26.  Motion  Sickness  Total  Severity  Box  Plots. 


Similar  comments  apply  to  the  nausea,  oculomotor,  and  disorientation  symptom 
sums  for  which  box  plots  are  plotted  in  Figures  27,  28,  and  29.  Again,  the 
distributions  are  largely  skewed  toward  low  values  except  for  the  HMD  in  the 
moving  vehicle.  Friedman  tests  show  significant  differences  among  treatments 
for  the  nausea  symptom  (chi-square  =  35.735,  df  =  7,  N  =  8,  p  <  .001),  oculomotor 
symptom  (chi-square  =  32.101,  df  =  7,  N  =  8,  p  <  .001),  and  the  disorientation 
symptom  (chi-square  =  20.824,  df  =  7,  N  =  8,  p  <  .004).  Again,  application  of 
Scheffe  pair-wise  comparison  tests  shows  that  these  differences  are  attributable 
to  the  HMD  in  the  moving  vehicle,  at  least  for  the  nausea  symptom  (Scheffe's  test 
statistic  =  21.125,  df  =  7,p  <  .01)  and  oculomotor  symptom  (Scheffe's  test  statistic 
=  18.000,  df  =  7,  p  <  .02). 
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Figure  27.  Motion  Sickness  Nausea  Symptom  Box  Plots 
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Figure  28.  Motion  Sickness  Oculomotor  Symptom  Box  Plots. 


Figure  29.  Motion  Sickness  Disorientation  Symptom  Box  Plots. 


A  review  of  the  16  motion  sickness  ratings  shows  that  the  distributions  are 
largely  skewed  from  normality  with  outliers;  this  is  especially  true  for  the  direct 
viewing  treatments  but  also,  in  many  cases,  for  the  other  treatments.  The  ratings 
show  significant  general  discomfort  (chi-square  statistic  =  35.238,  df  =  7,p<  .001) 
for  the  moving  vehicle  with  the  HMD  (Scheffe  test  value  =  15.125,  df  -  7,  p<  .05). 
Six  of  the  eight  participants  reported  discomfort  with  the  HMD  in  the  moving 
vehicle,  while  the  remaining  two  did  not.  Figure  30  is  a  box  plot  of  the  general 
discomfort  ratings  as  a  function  of  the  treatments,  which  shows  significant 
discomfort  for  the  moving  vehicle  with  the  HMD.  Although  the  plot  shows  an 
incident  of  slight  discomfort  for  the  stationary  vehicle  with  the  HMD,  no 
discomfort  was  reported  for  the  direct  vision  treatments. 

For  the  moving  vehicle  with  the  HMD,  significant  increases  are  reported  for 
nausea  {p  <  .001),  stomach  awareness  (p  <  .001),  dizziness  with  eyes  open 
(p  <  .003)  and  closed  (p  <  .003),  sweating  (p  <  .009),  burping  (p  <  .023),  and 
difficulty  in  concentration  (p  <  .024).  Although  the  box  plots  for  these  measures 
show  significant  increases  in  incidences  for  the  moving  vehicle  with  the  HMD, 
none  are  reported  for  the  other  treatments  except  for  sweating  and  concentration. 
Several  participants  reported  incidences  of  difficulty  in  concentration  for  direct 
viewing  in  the  moving  vehicle,  and  one  participant  reported  slight  difficulty  with 
the  HMD  in  the  stationary  vehicle.  In  contrast,  some  incidences  of  sweating  were 
reported  during  all  treatments  possibly  because  of  the  relatively  warm  weather; 
however,  more  incidents  were  reported  for  the  HMD  with  the  moving  vehicle. 
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The  remaining  components  of  motion  sickness  show  statistically  insignificant 
higher  ratings  for  the  treatment  of  the  HMD  in  the  moving  vehicle  with  fewer 
participants  reporting  incidences.  For  this  treatment,  several  participants 
reported  incidences  of  fatigue,  salivation,  fullness  of  head,  and  vertigo.  For  all 
treatments,  some  incidences  were  reported  of  headaches,  eye  strain,  difficulty  in 
focusing,  and  blurred  vision.  These  latter  measures  may  just  as  well  have  been 
caused  by  a  demanding  visual  search  with  optical  equipment  on  a  sunny  day; 
however,  more  incidents  were  reported  for  the  HMD  with  the  moving  vehicle. 


none  3D-snd  none  3D-snd  none  3D-snd  none  3D  -snd  cueing 

Direct _ HMD  Direct _ HMD  vision 

Stationary  Moving  vehicle 


Treatments 

Figure  30.  Motion  Sickness  General  Discomfort  Box  Plots. 

4.3.4  Selcon  &  Taylor's  Situation  Awareness  Rating  Technique  (SART) 
Situation  awareness  is  significantly  influenced  by  the  visual  treatments  as 
determined  by  analysis  of  the  first  factorial  component.  Figure  31  shows  a 
factorial  component  loading  diagram  for  the  ten  scales  after  reduction  to  three 
components  with  a  factor  analysis  using  principal  component  analysis  as  the 
extraction  method  (78.030%  total  variance  explained),  and  Varimax  rotation  with 
Kaiser  normalization.  As  noted  in  the  key  on  the  figure,  the  scales  are  shape 
coded  by  their  status  as  SA  demand,  supply,  or  understanding.  A  study  of  the 
rotated  component  matrix  for  the  factorial  plot  of  Figure  31  suggests  that  the  first 
factorial  component  reflects  the  demand,  the  second  component  understanding, 
and  the  third  component  supply. 
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Figure  31.  SART  Factorial  Component  Plot  in  Rotated  Space. 


Figure  32  shows  EDA  box  plots  of  the  first  factorial  component  for  the 
treatments.  The  plots  show  the  distributions  to  be  fairly  normal  with  closely 
equal  variances.  A  univariate  RM  ANOVA  test  of  within-subjects  effects  is 
significant  by  visual  treatments  (p  <  .011,  F  =  11.644,  df  =  1,  error  df  =  7); 
however,  the  other  treatments  and  interactions  are  not  significantly  different 
from  each  other. 


The  overall  rating  of  SA  is  not  significantly  different  across  treatments  according 
to  a  univariate  RM  ANOVA  test  of  within-subjects  effects.  The  overall  SA  rating 
is  the  difference  between  the  sum  of  the  ratings  for  the  supply  and  the  demand 
domains  added  to  those  for  the  understanding.  Figure  33  shows  EDA  box  plots 
of  the  SA  ratings  for  the  treatments. 

Although  the  changes  in  SA  are  insignificant,  the  demands  on  maintaining 
awareness  are  increased  with  the  use  of  the  HMD.  The  application  of  an  NP 
Friedman  ANOVA  test  by  ranks  to  the  sum  of  the  demand  domain  ratings  shows 
significant  differences  across  treatments  (chi-square  statistic  =  15.968,  df=  7, 
p  <  .025).  A  Scheffe  pair-wise  comparison  test  for  the  contrast  of  rank  means 
shows  that  this  difference  is  attributable  to  the  visual  treatments.  Figure  34 
shows  box  plots  for  the  demand  sum  and  Figures  FI-1  through  H-3  show  plots 
for  the  demand  dimensions.  The  plots  are  not  as  uniform,  with  large  differences 
in  variances  and  several  outliers.  Considering  the  separate  dimensions,  the 
ratings  show  significant  increases  in  instability  (chi-square  statistic  =  19.936, 
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df  =  7,  p  <  0.006)  and  complexity  (chi-square  statistic  =  16.245,  df  =  7,  p  =  .023)  for 
the  moving  vehicle  or  with  use  of  the  HMD. 


Figure  33.  SART  Situation  Awareness  Box  Plots. 
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Figure  34.  SART  Demand  Sum  Box  Plots. 


Although  situation  understanding  decreases  with  use  of  the  HMD,  there  is  an 
increase  in  supply  of  awareness.  The  sum  of  the  understanding  domain  ratings  is 
statistically  significant  across  treatments  (chi-square  statistic  =  15.728,  df  =  7, 
p  <  .028),  while  the  supply  domain  sum  is  closely  significant  (chi-square  statistic 
=  14.612,  df  =  7,  p  <  .041).  Figure  35  shows  box  plots  for  the  supply  sum,  and 
Figures  H-4  through  H-7  show  plots  for  the  supply  dimensions.  Similarly, 
Figure  36  shows  plots  for  the  understanding  sum,  and  Figures  H-8  through  H-10 
show  plots  for  the  understanding  dimensions.  Several  of  the  plots  for  the  supply 
and  understanding  are  skewed  from  normality  or  bound  by  the  maximum  value. 
Considering  the  separate  dimensions  of  the  questionnaire,  the  ratings  show 
significantly  less  familiarity  (chi-square  statistic  =  25.155,  df  =  7,p<  0.001)  for  the 
moving  vehicle  or  with  use  of  the  HMD.  Further,  more  concentration  (chi-square 
statistic  =  17.244,  df  =  7,p  <  0.016)  was  needed  for  the  HMD  than  direct  vision. 


In  general,  the  plots  for  the  questionnaire  dimensions  show  the  detrimental 
effects  on  SA  of  the  use  of  the  HMD  as  compared  to  direct  vision.  Considering 
the  awareness  demand,  there  is  an  increasing  trend  in  variability  along  with  the 
statistically  significant  increases  in  instability  and  complexity.  Again,  for 
awareness  supply,  there  are  increasing  trends  in  arousal  and  divided  attention 
and  a  decrease  in  spare  mental  capacity,  along  with  the  significantly  increased 
need  for  concentration.  Finally,  considering  situational  understanding,  there  are 
decreasing  trends  in  information  quality  and  quantity  along  with  the  significant 
decrease  in  familiarity. 
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SART  scores 


Figure  35.  SART  Supply  Sum  Box  Plots. 


Figure  36.  SART  Understanding  Sum  Box  Plots. 
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4.4  Usability  Evaluations 

The  participants  rated  the  four  display  configurations  (direct  viewing  or  HMD, 
with  or  without  audio  cueing)  as  to  their  usability  in  the  vehicle  mobility  modes 
(stationary  or  moving).  They  rated  direct  viewing  as  more  useful  than  the  HMD 
and  the  sound  localization  as  being  of  more  use  with  the  HMD  than  with  direct 
viewing.  A  univariate  RM  ANOVA  shows  significant  effects  (F  =  103.176, 
dfn  =  1,  dfd  =  6,  p  <  .001)  for  the  vision  mode  (p  <  .001)  and  the  interaction  of  the 
vision  mode  with  the  cueing  format  (p  <  .046).  Figure  37  is  a  plot  of  the  average 
ratings  as  a  function  of  the  treatment  conditions.  One  participant  who  had 
completed  the  experiment  did  not  have  time  to  perform  the  ratings  because  of  a 
previous  travel  arrangement. 


Figure  37.  Exit  Rating  Box  Plots. 

4.5  Family-wise  Significance 

The  statistically  significant  analyses  are  rank  ordered  by  the  (two-sided) 
probabilities  of  significance  in  Table  8,  along  with  the  corresponding  overall 
alpha  level  computed  by  the  Holm  procedure.  The  table  shows  that  family-wise 
significant  differences  result  among  the  treatments  for  the  analyses  of  the  task 
performance,  allocations  of  attention  for  detection  and  identification,  TLX 
workload,  SART  SA,  motion  sickness,  and  the  usability  evaluation.  The  analysis 
for  the  heart  rate  is  not  significant  at  the  0.05  family-wise  level. 
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Table  8.  Summary  of  the  Statistically  Significant  Results 


Task  Component 

Probability 

Holm  a  Levels 

1.  Driving  performance 

Overall 

0.0001 

0.0056 

time  to  detect 

0.0001 

0.0100 

Number  detected 

0.001 

0.0125 

time  to  identify 

0.001 

0.0167 

number  identified 

0.001 

0.0250 

number  identified  correctly 

0.001 

0.0500 

2.  Motion  sickness- 

First  factorial  component 

0.000 

0.0062 

total  severity 

0.000 

0.0125 

nausea  symptom 

0.000 

0.0167 

general  discomfort 

0.000 

0.0083 

nausea 

0.000 

0.0100 

stomach  awareness 

0.000 

0.0125 

sweating 

0.009 

0.0167 

oculomotor  symptom 

0.000 

0.0250 

disorientation  symptom 

0.004 

0.0500 

dizziness  (eyes  open) 

0.003 

0.0125 

dizziness  (eyes  closed) 

0.003 

0.0167 

3.  Usability  evaluation 

0.000 

0.0071 

4.  Attention  allocation  loading  for  identification- 

Second  factorial  component 

0.001 

0.0083 

auditory 

0.001 

0.0125 

5.  TLX  workload- 

First  factorial  component 

0.001 

0.0100 

overall  sum 

0.000 

0.0167 

demand  sum 

0.001 

0.0250 

mental 

0.001 

0.0167 

physical 

0.010 

0.0250 

temporal 

0.010 

0.0500 

interaction  sum 

0.001 

0.0500 

effort 

0.001 

0.0167 

performance 

0.001 

0.0250 

frustration 

0.001 

0.0500 

6.  Attention  allocation  loading  for  detection- 

First  factorial  component 

0.009 

0.0125 

auditory 

0.001 

0.0125 

psychomotor 

0.001 

0.0167 

7.  SART- 

First  factorial  component  - 

0.011 

0.0167 

For  the  significant  analyses,  the  table  lists  the  statistical  results  for  separate 
analyses  of  the  component  measures  using  the  Holm  procedure  to  control  the 
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(sub)  family-wise  0.05  alpha  level.  For  the  task  performance,  the  analyses  for  the 
detection  and  identification  times  and  numbers  are  family-wise  significant.  The 
allocation  of  attention  for  the  detection  task  is  significant  because  of  the  loading 
on  the  auditory  and  psychomotor  channels.  Similarly,  the  attention  allocation  for 
the  identification  task  is  significant  because  of  the  loading  on  the  auditory 
channel.  For  the  TLX  workload,  the  analyses  of  the  grand,  demand  and 
interaction  sums  are  significant,  as  are  those  of  all  components  for  the  sums. 
Although  the  SART  analysis  of  the  factorial  component  is  significant,  those  for 
SA  and  the  SA  demand,  supply,  and  understanding  are  not.  The  total  severity 
and  the  nausea,  oculomotor,  and  disorientation  symptoms  of  the  motion  sickness 
are  family-wise  significant,  as  are  the  components  listed  for  the  symptoms. 

Referring  to  the  previous  text,  the  performance  for  the  overall  task,  the  detection 
task,  and  the  identification  task  are  significantly  influenced  by  the  vehicle  motion 
and  the  vision  treatments.  The  time  to  detect  is  also  influenced  by  the  three-way 
Vehicle  Motion  x  Vision  Mode  x  Cueing  Treatments  interaction.  Similarly,  the 
questionnaires  are  statistically  influenced  by  the  vehicle  motion  and  the  vision 
treatments.  In  particular,  the  loading  factor  for  attention  allocation  of  the 
psychomotor  channel  for  the  detection  task  is  influenced  by  the  vision  treatment, 
while  the  auditory  channel  of  the  detection  and  identification  tasks  are 
influenced  by  the  cueing  treatment.  The  NASA-TLX  workload  components  are 
influenced  by  the  vision  treatments.  The  general  discomfort  component  of 
motion  sickness  is  induced  by  the  vehicle  motion  with  the  HMD.  Further,  the 
usability  evaluation  is  influenced  by  the  vision  treatment  and  the  interaction 
with  the  cueing  treatments.  Finally,  although  family-wise  insignificant,  the 
demand  components  of  instability  and  complexity  and  the  concentration  supply 
component  for  SA  are  influenced  by  the  vehicle  motion  and  the  vision 
treatments,  while  the  understanding  component  of  familiarity  is  influenced  by 
the  vision  treatment. 


5.  Discussion 


In  this  section,  the  effects  of  the  vision  systems,  localized  auditory  cueing,  vehicle 
motion,  and  environment  on  task  performance  and  mental  workload  are 
summarized.  Task  performance  is  reviewed.  Next,  mental  workload  as  measured 
by  attention  allocation,  perceived  workload  and  SA,  and  motion  sickness,  is 
discussed.  The  long-term  impact  of  mental  workload  on  performance  is  briefly 
reviewed.  Finally,  the  comments  by  the  participants  are  discussed. 

5.1  Treatment  Effects  on  Performance  and  Mental  Workload 

The  effects  of  the  vision  system,  sound  localization,  vehicle  motion  and  task 
environment  on  the  performance  and  mental  workload  are  reviewed. 
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5.1.1  Vision  System 

The  limited  FOV  and  resolution  of  the  indirect  vision  system  increased  the  visual 
workload  by  decreasing  the  target  visibility  and  the  search  glimpse  size.  The 
reduced  glimpse  size  increased  the  number  of  glimpses  needed  for  search  and 
the  cognitive  workload  needed  to  maintain  a  useful  search  pattern  over  the 
sector.  The  visual  performance  was  limited  by  the  resolution  and  FOV  of  the 
FIMD  used  in  this  study.  The  target  visibility  is  modulated  by  the  optical 
properties  of  the  indirect  visual  system,  which  are  restricted  by  the  FOV  of  the 
HMD.  At  30  degrees,  the  FOV  of  the  HMD  is  61%  of  the  48.8-degree  FOV  of  the 
camera,  and  the  scene  on  the  HMD  appears  expanded  1.63  times  that  of  the  real 
world,  making  the  targets  0.61  times  smaller  in  linear  size.  Further,  the  HMD 
with  180,000  rasters  has  68.8%  the  angular  resolution  of  the  camera  (768  x  494 
rasters).  For  these  reasons,  the  targets  on  the  HMD  appear  0.42  smaller  in  linear 
size  than  they  would  with  an  HMD  optically  matched  to  the  camera.  The 
electronic  camera  has  auto-focusing  with  automatic  white  balance  and  brightness 
control  maintained  by  an  auto-iris. 

5.1.2  Sound  Localization 

Sound  localization  with  the  intermittent  tones  increased  cognitive  workload  by 
interrupting  the  visual  search  to  allow  focusing  on  the  following  chirp.  In  this 
way,  the  acoustic  aiding,  which  was  meant  to  be  integrated  with  the  visual 
stimulus,  instead  became  a  secondary  task  competing  for  attentional  resources. 
Although  this  was  irritating  with  the  large  FOV  of  direct  viewing  requiring  only 
a  few  search  glimpses,  the  time  taken  to  evaluate  tones  was  less  than  that  needed 
for  the  multiple  glimpses  used  with  the  indirect  vision  system.  The  field  of 
search  was  limited  to  the  right  front  60-degree  sector  for  safety  reasons. 
Certainly,  sound  localization  has  proved  to  be  useful  in  target  search  with  an 
HMD  over  a  110-degree  front  (Nelson  et  al.,  1998).  We  would  expect  that  sound 
localization  would  be  helpful  during  direct  viewing  with  a  wider  field  of  search 
such  as  along  a  full  180-degree  front. 

The  effectiveness  of  the  sound  localization  was  further  limited  by  the  relative 
insensitivity  of  humans  to  synthetically  generated  stereophonic  sounds  through 
earphones.  The  literature  (Begault  &  Wenzel,  1990)  about  synthetic  sounds 
generated  in  earphones  reports  that  humans  have  a  roughly  20-degree 
localization  accuracy  in  both  azimuth  and  elevation.  The  localization  is  reported 
to  be  more  accurate  at  a  30-  to  60-degree  angle  to  the  side  of  the  head. 
Furthermore,  the  HRTF  used  in  the  Air  Force  3-D  sound  localization  system  is  a 
fixed  database  measured  from  a  manikin  standard.  However,  the  HRTF  of  an 
individual  is  determined  by  the  time  delay  and  amplitude  differences  in  the 
receipt  of  the  same  signal  at  the  two  ears,  which  can  vary  with  head  size  and  the 
fine  structure  of  the  pinna  of  the  outer  ear. 
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5.1.3  Vehicle  Motion 

Vehicle  motion  further  decreased  the  resolution  because  of  image  blurring; 
however,  for  some  participants,  motion  stereopsis  of  the  target  against  the 
variable  background  increased  target  visibility.  Vehicle  motion  dynamically 
changed  the  image  scene,  making  it  appear  unstable  and  increasing  the  cognitive 
workload.  More  concentration  was  needed  to  maintain  a  mental  model  of  the 
relation  of  the  viewer  to  the  SASPRF.  The  mental  model  is  needed  to  maintain  a 
useful  search  pattern;  this  is  especially  true  with  the  limited  FOV  of  the  HMD. 
Further,  there  was  an  increased  chance  of  losing  the  detected  target  during 
identification.  Cognitive  workload  was  increased  during  vehicle  motion  with  the 
HMD  that  tended  to  induce  motion  sickness.  This  forced  a  monitoring  of 
discomfort  awareness  as  a  secondary  task  competing  for  attentional  resources.  In 
some  cases,  motion  sickness  dominated  the  responses,  causing  the  participant  to 
momentarily  discontinue  the  primary  task  of  target  detection  and  identification; 
however,  none  of  the  participants  elected  to  stop  trial  runs. 

Motion  sickness  is  reported  (Yardley,  1992)  to  be  provoked  by  sensory  conflict 
between  the  visual  field  and  sensorimotor  activities,  which  involve  the  vestibular 
system  through  body  and  head  movements.  Since  the  participants  sat  on  the 
right  side  of  the  vehicle  and  the  camera  was  mounted  in  the  center,  the 
movements  seen  on  the  HMD  were  different  from  those  that  were  received 
physically  from  the  seat  through  the  vehicle  frame.  The  inconsistency  between 
the  visual  and  physical  stimuli  may  have  been  disconcerting  to  some,  especially 
while  the  camera  was  rotated  off  to  the  side  during  search  and  when  the  vehicle 
backed  during  the  return  drive  to  the  starting  line.  This  problem  may  have  been 
aggravated  further  by  latencies  in  the  responses  of  the  pan-and-tilt  mechanism 
that  resulted  in  the  camera  lagging  slightly  behind  the  head  movements. 

Another  source  of  motion  sickness  is  motion  blurring  of  the  image  on  the  LCDs 
during  side  view  searches  from  the  moving  vehicle.  This  results  in  the  display 
appearing  slightly  out  of  focus  because  the  video  return  is  blurred,  with  an 
accompanying  loss  of  dynamic  resolution.  In  some,  this  apparently  induces  a 
lack  of  convergence  accommodation,  resulting  in  blur-driven  asthenopia 
symptoms.  As  reported  in  the  literature,  motion  sickness,  especially  asthenopia 
symptoms  (Ebenholtz,  1992),  can  be  produced  by  insufficiencies  in  visual 
stimulation. 

5.1.4  Task  Environment 

The  task  environment  for  indirect  vision  tended  to  increase  sensitivity  to  motion 
sickness  because  of  sensory  degradation  while  participants  rode  in  the  enclosed 
compartment.  Although  the  direct  vision  target  search  was  performed  with 
unrestricted  natural  vision  from  an  open  cab,  the  indirect  vision  search  was  from 
an  enclosed  cab.  The  participants  experienced  physical  isolation,  darkness,  heat, 
and  noise  during  the  trial  runs  with  the  indirect  vision  system,  which  were 
different  from  conditions  for  direct  vision. 
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5.2  Task  Performance 

The  target  detection  and  identification  performances  are  of  interest.  The 
probabilities  for  target  detection  and  identification  and  associated  average  times 
are  plotted  in  Figures  38  through  42  as  a  function  of  the  treatments.  Table  9  lists 
the  values  for  these  plots.  The  figures  show  that  the  use  of  the  HMD  decreases 
the  probability  of  detection  and  increases  the  detection  time  for  those  targets  that 
are  detected.  Given  detection,  although  the  HMD  and  vehicle  motion  reduce  the 
probability  of  identification  and  increase  the  identification  time  for  targets  that 
are  identified,  vehicle  motion  decreases  the  probability  of  correct  identification. 
Figure  38  is  a  plot  of  the  probability  of  target  detection.  Figure  39  is  a  plot  of  the 
average  target  detection  times,  given  a  successful  detection.  Figure  40  is  a  plot  of 
the  conditional  probability  of  target  identification,  given  detection.  Figure  41  is  a 
plot  of  the  average  target  identification  times,  given  identification.  Finally, 
Figure  42  is  a  plot  of  the  conditional  probability  that  the  identification  is  correct. 

Table  9.  Statistically  Significant  Task  Performance 

Time  (seconds)  Targets  (percent) 

Configuration  Detect  Identify  Detect  Identify  Correct 

1.  Stationary 


1.1  Direct  viewing 


1.1.1  No  sound 

3.367 

4.39 

90.10 

94.79 

87.20 

1.1.2  Sound 

4.042 

4.39 

90.10 

94.79 

87.20 

1.2  HMD  viewing 

1.2.1  No  sound 

8.067 

5.62 

55.73 

42.99 

73.91 

1.2.2  Sound 

8.283 

5.62 

55.73 

42.99 

73.91 

2.  Moving 


2.1  Direct  viewing 


2.1.1  No  sound 

4.704 

6.46 

88.02 

85.21 

52.78 

2.1.2  Sound 

4.041 

6.46 

88.02 

85.21 

52.78 

2.2  HMD  viewing 

2.2.1  No  sound 

7.732 

7.01 

30.72 

30.51 

61.11 

2.2.2  Sound 

9.054 

7.01 

30.72 

30.51 

61.11 

Maximum  possible 

15.0 

10.0 

96 

No.  detected 

No.  identified 

Notes  on  statistical  significance: 

Time  to  detect  -  movement,  vision,  and  movement  by  vision  by  cueing. 

Targets  detected  -  movement,  vision,  movement  by  vision. 

Time  to  identify  -  movement,  vision,  movement  by  vision. 

Targets  identified  -  movement,  vision,  movement  by  vision. 

Correct  identification  -  movement,  vision 
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Probability  (%) 


100. 


Figure  38.  Probability  of  Target  Detection. 


Figure  39.  Average  Target  Detection  Time  (given  success). 
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Figure  40.  Conditional  Probability  of  Target  Identification  (given  detection). 


Figure  41.  Average  Target  Identification  Time  (given  success). 


Figure  42.  Conditional  Probability  That  Identification  is  Correct. 


5.3  Mental  Workload 

The  mental  workload  is  of  interest  since  it  can  influence  performance.  Summarized 
are  the  mental  workload  measures  as  determined  by  the  attention  allocation,  the 
perceived  task  workload,  SA,  and  motion  sickness. 

5.3.1  Measures 

Table  10  lists  the  average  scores  for  the  statistically  significant  mental  workload 
measures  as  a  function  of  the  pertinent  treatments.  The  maximum  possible  scores 
are  listed  for  reference.  Table  11  lists  the  average  scores  for  the  statistically 
significant  attention  allocation  factors  for  the  detection  and  identification  tasks. 
The  equivalent  verbal  anchors  are  listed,  along  with  the  scores  for  reference.  Note 
that  the  maximum  possible  score  on  an  attention  allocation  channel  is  7. 

5.3.2  Workload  Costs 

Over  long  periods  of  time,  excessive  mental  loading  can  lead  to  fatigue  and 
consequently  slow  responses  and  increase  errors.  Furthermore,  increased  mental 
workload  can  result  in  a  loss  of  SA  for  other  problems  in  the  environment,  which 
can  impact  future  decision  making.  The  increased  workload,  the  demand  on  SA, 
and  the  chance  of  motion  sickness  increase  the  psychological  stress  on  the  gunner 
with  the  indirect  vision  system.  In  a  two-person  vehicle,  the  gunner  may  be 
expected  to  perform  other  functions  such  as  navigation  and  communications.  At 
high  stress  levels,  the  resulting  physiological  arousal  is  detrimental  to 
performance. 
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Table  10.  Statistically  Significant  Mental  Workload  Questionnaire  Results 


Configuration 

Motion 

Sickness3 

TLX 

Workload 

SART 

Factorial 

1.  Stationary 

1.1  Direct  viewing 

4.41 

13.96 

18.82 

1.2  HMD  viewing 

4.41 

28.10 

23.28 

2.  Moving 

2.1  Direct  viewing 

4.41 

13.96 

18.82 

2.2  HMD  viewing 

22.19 

28.10 

23.28 

Maximum  possible 

120 

54 

N/A 

"Note:  No  trials  aborted  because  of  motion  sickness. 

Notes  on  statistical  significance: 

Motion  sickness  -  movement  by  vision  (HMD  with  motion) 
TLX  workload  -  vision 

SART  factorial  -  vision 


Table  11.  Statistically  Significant  Attention  Loading  Factors 


Configuration 

Value3 

Verbal  Anchor 

I.  DETECTION  TASK 

(a)  Visual  channel: 

All  treatments 

3.575 

"discriminate" 

(b)  Cognitive  channel: 

All  treatments 

3.367 

"recognize" 

(c)  Audio  channel: 

Non  cueing  treatments 

Sound  cueing  treatments 

0.062 

3.293 

"none" 

"orient,"  "verify" 

(d)  Psychomotor  channel: 

Direct  vision 

HMD  vision 

1.788 

3.781 

"toggle" 

"continuous,"  "manipulate" 

II.  IDENTIFICATION  TASK 

(a)  Visual  channel: 

All  treatments 

3.973 

"inspect" 

(b)  Cognitive  channel: 

All  treatments 

3.667 

"recognize" 

(c)  Audio  channel: 

Non  cueing  treatments 

Sound  cueing  treatments 

0.000 

2.725 

"none" 

"orient,"  "verify" 

(d)  Psychomotor  channel: 

All  treatments 

1 

3.039 

"continuous" 

"Note:  maximum  possible  on  any  channel  equals  7. 


57 


The  immediate  effect  is  to  produce  tunneling  or  narrowing  of  the  perceptual  and 
cognitive  attention  to  concentrate  on  the  detection  and  identification  tasks  while 
ignoring  the  surroundings.  There  is  a  tendency  for  working  memory  loss  with 
reduced  capacity  for  processing  new  or  complex  material,  along  with  a  focus  on 
well-learned  responses  and  a  rapid  execution  of  these  responses  at  the  expense  of 
possible  errors.  The  resultant  shedding  of  secondary  tasks  may  not  be  done  in  an 
optimal  order  for  successfully  coping  with  the  problem  environment.  In 
particular,  motion  sickness  distracts  from  the  task  since  the  discomfort  is 
intrusive  and  the  gunner  cannot  readily  concentrate.  Although  the  immediate 
task  performance  may  deteriorate,  the  long-term  effects  are  cumulative,  resulting 
in  accumulated  fatigue  over  time,  which  can  adversely  affect  performance  of 
subsequent  tasks  (Wickens,  Gordon,  &  Liu,  1998). 

5.4  Participants'  Comments 

The  comments  by  the  participants  agree  with  the  experimental  results.  The 
comments  are  listed  in  Table  12.  The  participants  felt  that  although  sound 
localization  worked  moderately  well,  the  sound  increased  the  detection  times  for 
direct  viewing  from  the  stationary  vehicle.  In  this  treatment,  they  primarily 
relied  on  sight  by  scanning  with  their  eyes  and  not  turning  their  heads.  However, 
the  sound  helped  orient  the  participants  toward  the  target  with  the  HMD  in  the 
stationary  vehicle.  Furthermore,  the  participants  reported  that  sound  localization 
helped  them  to  find  the  targets  from  the  moving  vehicle  with  both  direct  vision 
and  the  HMD. 


6.  Conclusions 


The  choice  of  HMD  that  was  used  in  this  study  with  its  reduced  resolution  and 
limited  FOV  severely  limited  the  detection  and  identification  of  targets.  Fewer 
targets  were  detected  and  identified,  and  it  took  longer  to  do  so  with  the  HMD 
compared  to  targets  processed  with  direct  vision.  Using  the  HMD  in  a  moving 
vehicle  further  reduced  task  performance.  Sound  localization  with  intermittent 
tones  increased  the  time  to  detect  for  direct  viewing  in  a  stationary  position  but 
decreased  the  time  for  the  HMD  in  a  moving  vehicle.  Similar  comments  apply  to 
identification  from  the  stationary  position;  however,  fewer  targets  were 
identified  from  the  moving  vehicle  with  sound  localization  than  without.  The 
advantages  of  auditory  cueing  for  target  detection  may  have  been  limited  by  the 
choice  of  an  intermittent  tone  for  sound  localization  and  the  restricted  search 
sector  used  in  the  experiment. 

Ratings  of  task  attention  loading  show  that  increased  attention  was  needed  for 
the  detection  task  with  audio  cueing,  especially  with  the  HMD.  This  is  also  true 
for  the  identification  task.  The  use  of  the  FIMD  increased  the  perceived  workload 
because  of  the  limited  FOV .  For  SA,  use  of  the  HMD  with  vehicle  motion  made 
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the  perceived  situation  more  unstable  and  complex,  decreased  familiarity,  and 
concurrently  increased  the  need  to  concentrate  on  the  task.  Finally,  the  use  of  the 
HMD  with  vehicle  motion  tended  to  induce  general  discomfort,  which  led  to 
motion  sickness  that,  in  turn,  increased  mental  workload.  Over  a  long  period  of 
time,  the  increase  in  workload  may  lead  to  fatigue  and  errors. 

Table  12.  Comments  by  Participants 


Stationary  vehicle 

(a)  Direct  viewing  - 

No  sound  cues 

Easy  to  observe  targets. 

Does  not  cause  stress. 

Less  distracting. 

3D-sound  cues 

Increases  detection  times. 

Still  primarily  relying  on  sight. 

Sound  works  moderately  well;  minimal  benefit. 

Scan  with  eyes  and  do  not  turn  head. 

(b)  HMD  - 

No  sound  cues 

Hard  to  observe  targets;  camera  is  jerky. 

Camera  is  not  clear  and  makes  your  eyes  strain. 

Low  resolution,  motion  sickness. 

3D-sound  cues 

Sound  helps  orient  you. 

Hard  to  observe  targets. 

Need  better  camera;  sound  cues  help. 

Moving  vehicle 

(a)  Direct  viewing  - 

No  sound  cues 

Easy  to  observe  targets;  would  stop  to  identify. 

Does  not  cause  stress. 

3D-sound  cues 

Sound  helps  orient;  increases  target  detection. 

Still  primarily  relying  on  sight. 

(b)  HMD  - 

No  sound  cues 

Extremely  hard  to  view  targets. 

Camera  vibrates  and  jumps. 

3D-sound  cues 
Sound  helps  orient. 

Sound  needs  to  be  louder  and  adjustable. 

Better  fitting  helmet  and  ear  phone  plugs. 

_ Sound  cues  needed  with  zoom  lens  for  both  moving  and  stationary. 
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7.  Recommendations  for  Further  Research 


We  recommend  continuing  studies  of  the  performance  with  HMDs,  since  the 
development  of  miniaturized  camera  systems,  high  performance  computers,  and 
low  cost,  high  resolution  HMDs  with  a  wide  FOV  will  in  the  future  provide  a 
more  natural  virtual  reality  display  environment  for  the  operator  of  ground 
military  vehicles.  ° 


We  recommend  investigating  the  advantages  of  combined  visual  and  sound 
localization  cues  for  target  detection  with  HMDs.  Although  the  audio  cue  will 
orient  the  gunner,  the  visual  cue  can  find  the  target  within  the  visual  field  with 
the  accuracy  of  the  target  database. 


The  use  of  subjective  questionnaires  to  measure  perceived  workload,  SA,  and 
motion  sickness  has  provided  insight  into  the  increases  in  mental  workload 
incurred  with  indirect  vision  targeting.  We  recommend  the  collection  of 
subjective  data  in  further  experiments  and  the  use  of  factorial  analysis  in  data 
analysis  of  questionnaires  to  control  Type  I  error. 

Motion  sickness  continues  to  be  a  concern  for  indirect  vision  operations,  and 
techniques  for  controlling  sickness  without  adversely  affecting  performance 
should  be  researched  further. 

We  recommend  further  investigations  into  modeling  task  performance  and  the 
effects  of  mental  workload  on  indirect  vision  operations.  A  point  of  particular 
concern  is  the  inclusion  of  motion  sickness,  including  the  sources  and  symptoms 
and  their  effect  of  performance. 

Since  future  vehicle  designs  will  include  multitasking  along  with  targeting,  we 
recommend  research  into  multiple  task  performance  involving  crew  interaction 
and  communications  and  navigation  functions  for  indirect  vision  activities.  The 
present  study  was  limited  to  targeting  and  did  not  consider  the  higher  cognitive 
functions  that  are  required  of  future  combat  vehicle  operators.  A  future  study 
should  include  automated  adaptive  aiding  for  the  performance  of  multiple  tasks 
in  future  combat  systems. 
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INTENTIONALLY  LEFT  BLANK 


APPENDIX  A 


FORMS  FOR  INFORMED  CONSENT  AND  QUESTIONNAIRES 


INTENTIONALLY  LEFT  BLANK 


Table  A-l.  Informed  Consent  Form 

Army  Research  Laboratory 
Human  Research  and  Engineering  Directorate 
Aberdeen  Proving  Ground,  MD 

Title  of  Research  Project:  Comparison  of  direct  vision  to  a  head-mounted  display, 
with  and  without  audio  cueing,  for  target  detection  and  identification  from  a 
stationary  and  a  moving  vehicle. 

Principal  Investigator:  Christopher  Smyth  Phone:  (410)278-5935 

SSCB,  SPD,  HRED,  ARL 
Aberdeen  Proving  Ground,  MD  21005-5425 

Location  of  Study:  SASPRF 

Aberdeen  Proving  Ground,  MD 


Purpose  of  Study 

You  are  invited  to  participate  in  a  target  detection  study  that  compares  direct  to 
indirect  vision.  In  both  conditions,  you  will  search  for  targets,  but  in  the  indirect 
vision  condition,  you  will  view  the  SASPRF  through  a  helmet-mounted  video 
display  that  is  receiving  an  image  from  a  head-slaved  video  camera.  The  camera 
is  mounted  on  the  roof  of  a  HMMWV.  This  work  is  being  conducted  to 
determine  the  problems  that  may  occur  when  vehicles  are  operated  indirectly 
from  externally  mounted  cameras.  The  U.S.  Army  is  interested  in  this 
information  because  it  plans  to  develop  armored  vehicles  that  can  be  driven  and 
operated  with  indirect  vision  systems  so  that  the  crew  members  will  be  better 
protected  from  enemy  fire. 

Procedure 

If  you  decide  to  participate  in  our  study,  one  of  our  administrators  will  describe 
the  details  of  the  procedure  to  you.  A  brief  description  is  as  follows:  You  will  be 
sitting  in  a  HMMWV  that  will  either  be  parked  on  a  target  facility  or  driven 
down  the  center.  For  the  indirect  search,  you  will  be  wearing  a  helmet  with  a 
head-mounted  display  that  will  show  you  the  scene  as  you  turn  your  head.  You 
will  be  asked  to  tell  the  experimenter  when  you  see  a  target.  For  part  of  the 
study,  you  will  hear  a  stereo  tone  that  will  point  you  in  the  direction  of  the 
target.  You  will  then  use  a  pair  of  binoculars  or  the  camera  in  zoom  to  identify 
the  target.  The  study  will  take  about  6  hours  with  an  hour  break  for  lunch.  There 
will  be  a  10-minute  rest  break  between  the  phases.  You  will  be  given  a 
questionnaire  to  answer  following  the  indirect  search.  The  questionnaire  will  ask 
for  your  impressions  of  the  visual  display  and  its  performance. 
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Risks 


There  are  minimal  risks  associated  with  this  experiment.  The  experiment  is  being 
conducted  on  the  SASPRF.  No  weapon  firing  will  be  allowed  during  this  study. 
However,  for  your  own  safety  you  should  obey  the  safety  officer,  who  will 
remain  next  to  you  throughout  the  study. 

A  potential  risk  concerns  motion  sickness  which  can  occur  in  some  people  while 
riding  in  enclosed  compartments.  Symptoms  of  motion  sickness  are  nausea,  cold 
sweating,  pallor,  and  vomiting.  You  can  stop  the  experiment  at  any  time  by 
telling  the  safety  officer,  and  if  you  feel  sick,  we  would  prefer  that  you  stop  the 
experiment.  Motion  sickness  bags  will  also  be  positioned  near  your  seat.  If  you 
vomit,  the  safety  officer  will  stop  the  experiment.  Also,  if  you  feel  or  become  sick, 
you  will  be  held  at  the  site  for  at  least  1  hour  until  your  symptoms  subside. 

For  your  own  safety,  you  should  wear  your  safety  belt  whenever  the  HMMWV  is 
moving.  In  addition,  you  will  receive  an  orientation  and  safety  briefing  before 
you  actually  begin  the  experiment. 
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Table  A-2.  Estimating  Attention  Allocation  Loading  Factors  Questionnaire 
for  Detection  and  Identification  Tasks 

Subject:  Date:  Time: 

Condition: 

Please  check  all  workload  elements  performed  during  this  test  condition. 


Visual-  none  detect 


0  1 


discriminate  track 

inspect  locate  read  scan 

--  I - x —  I - 1  — X - x  I - 1 

3  4  5  6  7 


selection 

Cognitive-  none  auto 

'  | - 1  -x - 1 - 

0  12 


recognize  evaluate 

judge  recall  estimate 

| - X —  I - X —  I  — X - I - X—  I 

3  4  5  6  7 


Auditory-  none  detect  orient 


0 


verify  discriminate 

focus  speech  interpret 

—  I  —x - xl - 1 - X— I 

4  5  6  7 


continuous  writing 

Psychomotor-  none  speech  toggle  manipulate  adjust  typing 

012  3456  7 
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Estimating  Workload  Scales 
For  allocating  attention  loading  factors 


1.  Visual  Loading- 

Scale  Value  Descriptor 
0.0  No  Visual  Activity 

1.0  Visual  Register/Detect  (detect  occurrence  of  image) 

3.7  Visual  Discriminate  (detect  visual  differences) 

4.0  Visual  Inspect/Check  (discrete  inspection /static  condition) 

5.0  Visual  Locate/Align  (selection  orientation) 

5.4  Visually  Track/Follow  (maintain  orientation) 

5.9  Visually  Read  (symbol) 

7.0  Visually  Scan/Search  Monitor  (continuous /serial  inspection,  multiple 
conditions) 


2.  Cognitive  Loading- 

Scale  Value  Descriptor 
0.0  No  Cognitive  Activity 
1.0  Automatic  (simple  association) 

1.2  Alternative  Selection 

3.7  Sign/Signal  Recognition 

4.6  Evaluation/Judgment  (consider  single  aspect) 

5.3  Encoding/Decoding,  Recall 

6.8  Evaluation/Judgment  (consider  several  aspects) 
7.0  Estimation,  Calculation,  Conversion. 


3.  Auditory  Loading- 

Scale  Value  Descriptor 
0.0  No  Auditory  Activity 

1.0  Detect /register  sound  (detect  occurrence  of  sound) 

2.0  Orient  to  Sound  (general  orientation. /attention) 

4.2  Orient  to  Sound  (selective  orientation. /attention) 

4.3  Verify  Auditory  Feedback  (detect  occurrence  of  anticipated  sound) 

4.9  Interpret  Semantic  Content  (speech) 

6.6  Discriminate  Sound  Characteristics  (detect  auditory  differences) 
7.0  Interpret  Sound  Patterns  (pulse  rates,  etc.) 


4.  Psychomotor  Loading- 

Scale  Value  Descriptor 

0.0  No  Psychomotor  Activity 
1.0  Speech 

2.2  Discrete  Actuation  (button,  toggle,  trigger) 

2.6  Continuous  Adjustment  (movement  control,  sensor  control) 

4.6  Manipulative 

5.8  Discrete  Adjustive  (rotary,  vertical  thumbwheel,  level  position) 
6.5  Symbolic  Production  (writing) 

7.0  Serial  Discrete  Manipulation  (keyboard  entries) 
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Table  A-3.  NASA-TLX  Workload  Rating  Form 


Subject:  Date:  Time: 

Test  Condition: 

Mental  Demand:  How  much  mental  and  perceptual  activity  was  required  (e.g. 
thinking,  looking,  searching,  etc.)?  Was  the  task  easy  or  demanding,  simple  or 
complex,  exacting  or  forgiving? 

low  I  —  I  —  I  —  I  —  I  —  I  —  I  —  I  —  I  —  I  high 


Physical  demand:  How  much  physical  activity  was  required  (e.g.  pushing, 
pulling,  turning,  controlling,  activating,  etc.)?  Was  the  task  easy  or  demanding, 
slow  or  brisk,  slack  or  strenuous,  restful  or  laborious? 

low  I  —  I  —  I  —  I  —  I  —  I  —  I  —  I  —  I  —  I  high 


Temporal  demand:  How  much  time  pressure  did  you  feel  due  to  the  rate  or  pace 
at  which  the  task  or  task  elements  occurred?  Was  the  pace  slow  and  leisurely  or 
rapid  and  frantic? 

low  I  —  I  —  I  —  I  —  I— -I  —  I  —  I— -I  —  I  high 


Effort:  How  hard  did  you  have  to  work  (mentally  and  physically)  to  accomplish 
your  level  of  performance? 

low  I  —  I  —  I  —  I  —  I  —  I  —  I  —  I  —  I  —  I  high 


Performance:  How  successful  do  you  think  you  were  in  accomplishing  the  goals 
of  the  task  set  by  the  experimenter  (or  yourself)?  How  satisfied  were  you  with 
your  performance  in  accomplishing  these  goals? 

good  I  —  I  —  I  —  I  —  I  —  I  —  I  —  I  —  I  —  I  poor 


Frustration:  How  insecure,  discouraged,  irritated,  stressed  and  annoyed  versus 
secure,  gratified,  content,  relaxed  and  complacent  did  you  feel  during  the  task? 

low  I  —  I  —  I  —  I  —  I  —  I  —  I  —  I  —  I  —  I  high 
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Subject: 

Test  Condition: 


able  A-4. 


Estimating  Motion  Sickness  Questionnaire 
Date:  Time: 


Please  rate  the  following  measures  of  motion  sickness: 


General  Discomfort 

None 

Slight 

Moderate 

Severe 

Fatigue 

None 

Slight 

Moderate 

Severe 

Headache 

None 

Slight 

Moderate 

Severe 

Eyestrain 

None 

Slight 

Moderate 

Severe 

Difficulty  Focusing 

None 

Slight 

Moderate 

Severe 

Increased  Salivation 

None 

Slight 

Moderate 

Severe 

Sweating 

None 

Slight 

Moderate 

Severe 

Nausea 

None 

Slight 

Moderate 

Severe 

Difficulty  Concentrating 

None 

Slight 

Moderate 

Severe 

Fullness  of  Head 

None 

Slight 

Moderate 

Severe 

Blurred  Vision 

None 

Slight 

Moderate 

Severe 

Dizzy  (eyes  open) 

None 

Slight 

Moderate 

Severe 

Dizzy  (eyes  closed) 

None 

Slight 

Moderate 

Severe 

Vertigo  * 

None 

Slight 

Moderate 

Severe 

Stomach  Awareness  ** 

None 

Slight 

Moderate 

Severe 

Burping 

None 

Slight 

Moderate 

Severe 

*  Vertigo  is  experienced  as  loss  of  orientation  with  respect  to  vertical  upright. 

**  Stomach  awareness  is  usually  used  to  indicate  a  feeling  of  discomfort  which  is 
just  short  of  nausea. 
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Table  A-5.  Situation  Awareness  Rating  Questionnaire 


Subject:  Date:  Time: 

Test  Condition: 


Please  rate  the  following  measures  of  situation  awareness: 


DEMAND 


Instability  of  situation: 

low  1 — 

- 2 - 

-3  -4~ 

5 

— 6 - 

-7 

high 

Variability  of  situation: 

low  1 — 

"2 - 

-3 - -4— 

5 

6 

-7 

high 

Complexity  of  situation: 

low  1 — 

- 2- - 

-3 - 4 — 

— 5 — 

— 6- - 

-7 

high 

SUPPLY 


Arousal: 

low 

1- 

- 2- - 

-3— 

4 

5 - 

"6 - 

-7 

high 

Spare  Mental  Capacity: 

low 

1- 

2 - 

-3 — 

— 4 — 

5 - 

-6 

-7 

high 

Concentration: 

low 

1— 

- 2 - 

-3— 

-4- 

— 5 - 

-6 - 

-7 

high 

Division  of  attention: 

low 

1~ 

2 

-3 — 

— 4 — 

5 

-6- — 

-7 

high 

UNDERSTANDING 


Information  Quantity:  low  1 - 2 - 3 - 4 - 5 - 6 - 7  high 

Information  Quality:  low  1 - 2 - 3 - 4 - 5 - 6 - 7  high 

Familiarity:  low  1 - 2 - 3 - 4 - 5 - 6 - 7  high 
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Table  A-6.  Usability  Rating  Questionnaire 

Usability  Rating 

Subject:  Date:  Time: 


Please  rate  the  four  target  detection  and  identification  systems  as  to  their 
usability  for  the  military  and  write  a  brief  rational  for  your  choice.  Remember 
that  you  are  comparing  the  four  systems  to  each  other  under  the  two  conditions 
of  vehicle  stationary  and  moving.  Please  use  this  opportunity  to  make 
recommendations  for  improvements  in  the  designs: 


I.  Stationary  Vehicle: 

Direct  Visual  not  use  would  use 

w/  binoculars  I - 1 - 1 - 1 - 1 - 1 _ I _ I 

&  no  sound  0  1234567 

Reason: 


Direct  Visual  not  use  would  use 

w/  binoculars  I - 1 - I - 1 - 1 - | - 1 _ | 

&  sound  cues  0  1  2  3  4  5  6  7 

Reason: 


HMD  &  camera  not  use  would  use 

w/  Zoom  I - 1 - 1 - 1 - 1 - 1 - 1 - 1 

&  no  sound  01234567 

Reason: 


Hh4D  &  camera  not  use  would  use 

w/  Zoom  I - 1 - 1 - 1 - 1 - 1 - 1 - 1 

&  sound  cues  01234567 


Reason: 
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II.  Moving  Vehicle: 


Direct  Visual  not  use  would  use 

w/  binoculars  I - 1 - 1 - 1 - 1 - 1 - 1 - 1 

&  no  sound  0  12  34567 

Reason: 


Direct  Visual  not  use  would  use 

w/  binoculars  I - 1 - I - 1 - I - 1 - 1 - 1 

&  sound  cues  0  12  34567 

Reason: 


HMD  &  camera  not  use  would  use 

w/  Zoom  I - 1 - 1 - 1 - 1 - 1 - i - 1 

&  no  sound  012  3456  7 

Reason: 


HMD  &  camera  not  use  would  use 

w/  Zoom  I - 1 - 1 - 1 - 1 - 1 - 1 - 1 

&  sound  cues  012  3456  7 

Reason: 
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PRELIMINARY  STUDY:  TARGET  DISTANCE 


The  preliminary  portion  of  the  study  determined  the  detection  probabilities  for 
the  two  vision  modes  in  a  stationary  position.  These  data  were  needed  to  design 
the  site  for  the  mobility  studies.  The  participant  was  seated  inside  the  vehicle 
parked  at  the  firing  line  centered  on  a  lane  of  targets,  facing  down  range.  The 
participant  was  asked  to  detect  a  target  when  it  was  presented  within  his  FOV; 
he  did  not  rotate  his  head  from  the  viewing  direction.  The  participant  started 
each  trial  facing  down  range  with  his  eyes  closed  and  his  head  down  before  the 
target  was  displayed.  The  experimenter  then  activated  a  target  with  a  toggle 
switch  and  a  brief  tone  to  alert  the  participant.  The  participant  looked  down 
range  and  activated  a  switch  if  a  target  was  present,  and  if  so,  verbally  estimated 
the  distance.  The  participant  had  10  seconds  to  respond,  followed  by  5  seconds' 
preparation  for  the  next  trial.  Each  target  was  presented  at  a  distance  from  25 
meters  to  400  meters  from  the  observation  point  in  25-meter  intervals  to  100 
meters,  50-meter  intervals  to  300  meters,  and  100  meters  to  400.  In  one  of  four 
trials,  no  target  was  presented  to  ascertain  the  participant's  natural  false  alarm 
rate. 

The  investigation  was  conducted  in  three  sessions  with  the  participant  evaluated 
in  the  first  session  with  direct  vision  and  then  in  the  second  session  with  the 
indirect  HMD  vision  system.  This  was  followed  by  another  direct  vision  session 
to  confirm  training  stability  in  a  standard  baseline,  intrusion,  retum-to-baseline 
format.  All  sessions  included  three  training  trials  followed  by  37  experimental 
trials,  including  10  with  no  targets.  The  target  sequence  including  no-target 
events  were  random;  in  this  way,  the  participant  saw  each  of  the  nine  settings 
three  times.  The  participant  was  allowed  a  10-minute  break  between  sessions. 

The  experimental  design  for  the  first  phase  is  a  fixed  factorial  3x9  experiment 
with  repeated  measures  on  the  vision  mode  and  target  distance  and  subjects  as  a 
random  factor.  The  experimental  hypothesis  is  that  the  targets  will  be  detected  at 
greater  distances  with  natural  vision  because  of  the  increased  visual  acuity  than 
with  the  HMD.  The  independent  variables  are  (1)  the  vision  modes  and  (2)  the 
target  distance.  The  dependent  variables  are  the  number  of  targets  detected  and 
the  times  to  detect,  if  detected.  The  experimental  results  of  this  investigation  are 
used  to  compute  descriptive  statistics  for  the  detection  distances  as  a  function  of 
distance  for  the  two  vision  modes.  The  baseline  and  retum-to-baseline  direct 
vision  data  are  compared  to  confirm  training  stability.  A  10-second  time  limit 
was  assigned  to  targets  not  detected. 

The  total  detection  times  were  analyzed  in  a  3x9  factorial  repeated  measures, 
parametric  ANOVA  as  a  function  of  target  distance  and  vision  mode.  To 
approximate  normal  frequency  distributions,  the  times  were  transformed  with 
the  natural  logarithm  before  the  statistical  analysis.  The  computed  F-values  for 
the  vision  mode  and  distance  main  effects  are  statistically  significant  at  the  0.001 
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level  and  their  interactions  are  statistically  significant  at  the  0.01  level.  This  is 
true  for  the  stringently  critical  F-distribution  values  (Winer,  p.  864)  that  follow 
from  the  application  of  the  Geisser-Greenhouse  corrections  to  the  degrees  of 
freedom  (Keppel,  1982,  p.  470).  The  corrections  are  applied  to  account  for  any 
bias  that  may  result  from  correlations  between  multiple  measures  obtained  from 
the  same  subjects  in  a  repeated  measures  experiment.  For  statistical  convenience, 
maximal  heterogeneity  for  the  variances  of  differences  is  assumed.  The  within- 
treatment  variances  satisfy  Cochran  s  test  for  homogeneity.  The  Cochran,s  test 
ratio  (Winer,  p.  208)  computed  as  the  ratio  of  the  maximum  cell  variance  to  the 
sum  of  variances,  is  less  than  the  critical  test  value  (Winer,  p.  876)  at  the  95% 
probability  level.  Of  course,  homogeneity  of  variances  is  a  necessary  condition 
for  the  unbiased  application  of  a  parametric  test. 


An  application  of  the  post  hoc  Scheffe's  two-way  comparison  test  shows  that  the 
mean  detection  times  are  not  significantly  different  for  the  baseline  and  return- 
to-baseline  direct  vision  tests.  These  results  show  that  additional  training  did  not 
occur  during  the  intervention  phase.  However,  the  means  for  the  indirect  vision 
(HMD)  intrusion  are  significantly  larger  then  those  for  the  direct  vision  at  the 
0.001  level.  Furthermore,  the  mean  times  for  the  150-  to  400-meter  set  of  ranges 
for  the  FIMD  are  significantly  larger  than  the  mean  times  at  the  0.001  level.  The 
Scheffe  test  (Keppel,  1982,  p.  151)  allows  multiple  comparisons  of  means  to  be 
made  without  changes  in  the  level  of  statistical  significance. 

The  data  for  the  false  alarm  rate  of  each  subject,  as  determined  from  the  number 
of  non-targets  falsely  detected  for  the  vision  modes,  are  too  sparse  for  statistical 
analysis.  However,  the  false  alarm  rate  was  practically  nil  for  the  direct  vision, 
while  4  of  8  subjects  detected  one  or  two  false  targets  in  ten  possibilities  in  the 
indirect  (HMD)  mode.  In  all  cases,  the  subjects  recognized  that  the  detection  was 
false  when  the  target  did  not  drop  after  their  button  push,  and  they  reported  that 
fact  to  the  experimenter. 


The  probability  of  detection  and  the  mean  target  detection  times  are  of  interest 
for  the  baseline  and  intrusion  phases  as  a  function  of  distance.  The  standard 
deviation  and  confidence  range  are  not  computed  for  the  150-  to  400-meter  set  of 
the  indirect  vision  (FIMD)  mode,  since  missed  targets  occurred  in  that  set.  The 
statistics  show  agreement  with  the  results  of  the  Scheffe's  two-way  comparison 
tests.  The  data  show  that  the  baseline  and  return  phase  detection  times  are  not 
significantly  different,  since  the  times  for  the  return  phase  are  within  the  95% 
confidence  band  for  the  baseline  times.  This  is  true  for  all  but  the  400  meters  in 
the  return  phase  where  the  lower  average  for  the  return  suggests  a  training  effect 
at  this  distance.  In  contrast,  while  the  mean  times  for  the  25-  to  100-meter  set  of 
the  intrusion  phase  are  within  the  baseline  confidence  band,  those  times  for 
targets  beyond  100  meters  continue  to  increase  above  the  upper  boundary  of  the 
confidence  band  with  increasing  distance. 
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In  summary,  the  analysis  shows  that  there  is  practically  no  effect  of  the 
experimental  order  on  the  detection  performance  with  direct  vision.  The  average 
detection  times  for  the  return  direct  vision  are  within  the  95%  confidence 
intervals  for  the  baseline  to  the  400-meter  targets  where  there  is  an  improvement 
because  of  training.  Further,  the  analysis  shows  that  the  performance  of  the 
direct  vision  and  indirect  HMD  are  statistically  the  same  to  100  meters.  However, 
the  performance  of  the  systems  diverges  for  distances  beyond  100  meters,  with 
the  performance  with  the  HMDs  decreasing  drastically  with  increasing  distance 
because  of  the  lower  resolution. 
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PRELIMINARY  STUDY:  TARGET  BEARING 


The  second  phase  of  the  preliminary  portion  determined  the  search  times  as  a 
function  of  the  angular  offset  for  the  two  vision  modes.  The  24  targets  used  in 
this  investigation  were  presented  one  at  a  time  in  a  random  order  at  a  distance  of 
50  meters  to  100  meters  from  the  participant's  position  across  a  170-degree  visual 
front.  These  distances  are  within  the  99%  probability  detection  distance  as 
determined  from  the  data  collected  in  the  first  phase.  The  targets  are  grouped  for 
data  analysis  by  four  45-degree  sectors  about  the  forward  direction.  The  sectors 
are  balanced  in  regard  to  target  disposition,  with  each  sector  containing  two 
targets  each  at  the  50-,  75-,  and  100-meter  distances  in  a  seemingly  random 
pattern. 

The  participant  was  seated  inside  the  vehicle  parked  at  the  center  of  the  firing 
line  facing  down  range.  The  participant  was  asked  to  find  targets  presented  by 
searching  with  head  movements  across  the  visual  front.  The  participant  started 
each  trial  facing  down  range  with  his  eyes  closed  and  his  head  down  before  a 
target  was  displayed.  He  was  told  the  direction  to  search  ("right"  or  "left");  the 
target  was  then  activated  by  the  experimenter,  and  the  participant  began 
searching  for  the  raised  target.  The  participant  pressed  a  button  for  automatic 
data  recording  and  target  depression  when  he  had  found  the  target  and  then 
verbally  reported  his  estimate  of  the  angular  offset.  The  participant  had 
15  seconds  to  give  his  report,  followed  by  5  seconds'  preparation  for  the  next 
trial. 

The  investigation  was  conducted  in  three  sessions  with  the  participant  evaluated 
in  the  first  session  with  direct  vision  and  then  in  the  second  session  with  the 
indirect  vision  system.  The  two  sessions  were  followed  by  another  direct  vision 
session  to  confirm  training  stability.  Each  session  block  had  32  trials,  with  the  24 
targets  each  presented  once  and  no  target  presented  in  8  trials  to  determine  the 
false  alarm  rate.  The  target  sequence  was  random  for  each  block,  and  all 
distances  and  directions  were  equally  likely;  the  no-target  events  were  randomly 
distributed  across  this  sequence.  The  participants  were  allowed  a  10-minute 
break  between  session  blocks. 

The  design  for  the  second  phase  is  a  3x24  fixed  factorial  experiment  with 
repeated  measures  on  the  vision  mode  and  the  target  and  with  subjects  as  a 
random  factor.  The  target  distance  and  bearing  are  random  factors  because  of  the 
random  presentation  order  of  the  targets.  The  experimental  hypothesis  is  that  the 
targets  will  be  detected  sooner  with  natural  vision  because  of  the  wider  FOV 
than  with  the  indirect  vision  HMD.  The  independent  variables  are  (1)  the  vision 
modes  and  (2)  the  target  set.  The  dependent  variable  is  the  time  to  detection. 

The  mean  time  to  detect  each  target  was  plotted  as  a  function  of  offset  angle  for 
the  baseline  and  intrusion  vision  mode.  The  times  were  averaged  across  the  eight 
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subjects  and  include  the  15-second  time  limit  when  a  target  was  not  detected. 
The  regression  trend  analysis  of  the  target  detection  data  as  a  quadratic  function 
of  the  angular  offset  was  plotted  for  the  baseline  and  intrusion  modes.  The 
analysis  considered  all  target  presentations,  including  those  that  were  not  seen 
within  the  15-second  time  limit.  Also,  trend  plots  were  plotted  for  just  the 
detected  targets;  the  target  presentations  that  were  not  seen  by  the  participants 
were  not  included  in  this  analysis.  A  comparison  of  the  two  sets  of  plots  suggests 
that  the  data  fall  within  two  groups,  detected  set  and  outliers. 

The  source  of  the  outliers  was  deducted  from  the  mean  detection  times  and  the 
probability  of  detection  for  each  target  as  a  function  of  vision  mode.  The  data 
show  that  at  least  six  targets  had  a  low  probability  of  detection  during  the 
intrusion  mode.  A  check  of  the  facility  showed  that  these  targets  were  to  the 
sides  against  a  dark  brown  woodland,  unlike  those  to  the  front,  which  were  seen 
against  a  brighter  yellow-green  field  grass.  Since  the  objective  of  this  experiment 
is  to  determine  the  detection  time  as  a  function  of  offset  angle  for  equally  visible 
targets,  the  target  presentations  with  time-limited  measures  are  dropped  from 
the  analysis. 

A  multiple  regression  analysis  was  applied  to  the  data  with  categorical  and 
continuous  variables  for  a  repeated  measures  design  with  coding  of  subjects  by 
criterion  scaling  and  vision  mode  by  dummy  coding  as  the  fixed  factor  (baseline 
or  intrusion).  The  distance  and  angle  of  the  targets  are  interpreted  as  random 
continuous  variables  because  of  the  random  order  of  presentation  and  coverage 
over  the  site.  The  analysis  compared  the  baseline  to  the  return  and  the  baseline  to 
the  intrusion  for  the  targets  detected.  The  analysis  shows  that  the  significant 
differences  (0.001  level)  in  the  detection  times  for  the  baseline  and  return  are 
attributable  to  the  distance,  offset,  and  subjects  but  not  to  the  mode.  In  contrast, 
the  significant  differences  in  the  detection  times  for  the  baseline  and  intrusion  are 
attributable  to  the  mode,  distance,  and  offset  but  not  to  the  subjects. 

The  data  for  the  false  alarm  rate  of  each  subject,  as  determined  from  the  number 
of  non-targets  falsely  detected  for  the  vision  modes,  are  too  sparse  for  statistical 
analysis.  However,  the  false  alarm  rate  was  practically  nil  for  the  direct  vision 
(3.1%),  while  4  of  8  subjects  detected  one  to  two  false  targets  in  ten  possibilities 
each  in  the  indirect  (HMD)  mode  for  an  overall  false  alarm  rate  of  14.1%.  In  all 
cases,  the  subjects  recognized  that  the  detection  was  false  when  the  target  did  not 
drop  after  their  button  push,  and  they  reported  that  fact  to  the  experimenter. 

In  summary,  the  analysis  shows  that  there  is  practically  no  effect  of  the 
experimental  order  on  the  detection  performance  with  direct  vision.  The  mean 
detection  times  increases  with  target  bearing.  However,  the  detection  times  are 
longer  with  the  indirect  vision  system,  for  which  the  mean  detection  time  does 
not  exceed  10  seconds. 
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Stationary  Moving  Treatments  Stationary  Moving  Treatments 

Figure  D-3.  Auditory  Channel  Attention  Allocation.  Figure  D-4.  Psychomotor  Channel  Attention  Allocation. 
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BOX  PLOTS  FOR  IDENTIFICATION  TASK  ATTENTION 
ALLOCATION  QUESTIONNAIRE  SCORES 
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Stationary  Moving  Treatments  Stationary  Moving  Treatments 

Figure  E-3.  Auditory  Channel  Attention  Allocation.  Figure  E-4.  Psychomotor  Channel  Attention  Allocation. 
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BOX  PLOTS  FOR  PERCEIVED  WORKLOAD 
QUESTIONNAIRE  SCORES 
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BOX  PLOTS  FOR  SITUATIONAL  AWARENESS 
QUESTIONNAIRE  SCORES 
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Stationary  Moving  Treatments 

Figure  G-3.  SART  Complexity  Demand. 
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Figure  G-6.  SART  Concentration.  Figure  G-7.  SART  Division  of  Attention. 
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Figure  G-8.  SART  Information  Quantity.  Figure  G-9.  SART  Information  Quality. 
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